Ethics in AI — a service design perspective

How much do we need to understand the technology? Pt. 1

There is a lot of ignorance around the topic of AI; if you’ve been to a AI event or read the marketing gumpf around a new AI-powered product recently, you’ve probably encountered something similar to this:

A robot arm (left), Wit.Ai chatbot code (middle) and Nadine the robot receptionist (right)
  • Neural network is “a computer system modelled on the human brain and nervous system”
  • Deep learning “is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks”. Again, examples are algorithms powering things like self-driving-cars.
  • Supervised versus unsupervised learning: “In supervised learning, the output datasets are provided which are used to train the machine and get the desired outputs whereas in unsupervised learning no datasets are provided, instead the data is clustered into different classes”.
  • Big data is simply lots of data that can be harnessed using the technology above.

How much do we need to understand the technology? Pt. 2

How much do customers need to understand the technology?

A logical follow-up question is how much do your customers/users need to understand the technology, specifically how you’re using it to provide service? We all have an idea that Google does some clever stuff to make search good, but do we need to know why search result 1 places above search result 2? Arguably not. But if you lost out on preferential health treatment to a seemingly identical patient due to some algorithmic calculation, you’d want to know why.

Vague and assumptive explainer from Facebook
Nvidia and their Drive PX platform, which is finding human readable ways to show AI thinking

To what cost?

In his excellent piece on the ‘myth of superhuman AI’, Kevin Kelly examines the need for ‘wetware’, or software designed to work like our own ‘wet’ biology:

DoNotPay helping with a PPI claim


AI is developing at such a brilliant pace it’s hard to draw a meaningful conclusion other than the importance of remaining curious. While the biggest leaps in AI will be made by tech giants and (probably) secretly deployed by governments, we in the design community have immense influence over how it manifests in our daily lives. Our critical thinking around AI will determine how much control we retain over it, and our knowledge will ensure it doesn’t simply ‘happen’ to us.

Further reading and other random bits

If you’re interesting in the topics above I’d recommend the following:

AI lols from xkcd

Some other questions I’m wrestling with:

AI as colleague — when do we need to start considering AI as a colleague? A sensible point would be when our use of AI goes beyond our human comprehension and we need some kind of translation. How will already messy human dynamics like reward and collaboration cope when a new colleague who never sleeps, demands payment or shows emotion joins the team?



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ross Breadmore

Ross Breadmore

Mum asked for a baby, dad asked for a transformer - I was the compromise. Chief product officer at 4G Capital.