My experience leads me to believe there are four important stages in designing an AI product. Much focus is given to the technology used to implement AI, but that is actually only the third step of a four step process. Put simply these steps are to 1) Select a problem, 2) Curate your context, 3) Implement your algorithm, 4) Act on the output. This all sounds straightforward enough, but the decisions made in steps 1, 2, and 4 (Select, Curate, and Act) can determine much more than any technical wizardry in step 3 (Implement).
The first step is to identify which problem will be best solved by AI. Many of the most famous AI research groups and companies in the world are so famous because they have chosen very carefully which task to tackle. Someone reading news from the AI community may be forgiven for thinking that AI is developing so that it can solve any problem thrown at it, but this is the result of two forms of survivorship bias: first, AI projects with unimpressive results are rarely reported; second, AI practitioners are getting better at not even attempting projects which are not likely to be solvable with AI.
A well chosen problem is one which paves the way for the next three steps, it will have a source of relevant data, a suitable algorithm will exist, and the algorithm's output will clearly define a suitable action. If the results, when demonstrated, are exciting to an audience then all the better.
Some of the biggest names in AI research are successful because of their dedicated scientists, but also famous because they cleverly choose which problem to work on and are good at promoting the results. Consider for example, Stylistic Transfer, the ability to take an image with a particular style, and apply that style to another image. Demonstrations of these projects were very popular around 2015-2018; you can turn your profile picture into a portrait by van Gogh, or watch a holiday video like it was an animation by Picasso, I even saw a video of a horse that appeared to be a zebra. These are all wonderful products in their own right, but I question whether these abilities were ever in demand before someone realised they were possible. Furthermore, these fantastic abilities do not guarantee that AI will have similarly fantastic powers when it comes to a problem you need solved.
The second step is to identify which data you will use as input to the algorithm, how you will collect it and, critically, what you will not collect. The received wisdom is that more data is better, but more data brings with it three distinct problems. 1) Storing the data. The least of your worries, it's a technical problem with many technical solutions. 2) Edge cases. A much nastier problem, the more data you have the more likely it is that your dataset contains some very rare occurrences. If you decide to address these issues you will find yourself spending a ever-increasing amount of resources on solving a never-ending succession of increasingly unlikely events. A much better solution is to declare precisely the events which your system will cater for and by implications those it will not. 3) Spurious correlations. The more data your collect, the more likely it is to contain patterns that are purely coincidence. For example, you might notice that the best advertising campaigns launch on months with 31 days, this may or may not be a meaningful causal relationship. Another spurious correlation is one which is revealing of underlying bias in the data collections. For example, recruitment AI often bias heavily towards hiring males, as that is what was done historically.
Curating the context therefore is a balancing act between your ambitions and your reach. Deploying a system to solve a small problem which could have solved a much more complex problem is a waste of resources, but a system which is used to confidently make decisions beyond its ability can lead to catastrophe. There is no shame in an AI system which does a small, well defined, task well. Finally, there is no "correct" or objectively optimal solution, the decision as to whether your project should use gender data, for example, is personal and political, not technical.
The third step is to choose the technology. There are many options and I won't go into them here. In many ways this is the easy bit. Software can be complicated but it is largely free of philosophical quandaries, infinitely complex situations, and politics.
The fourth step is to decide how you will use the information that your system outputs. If your system estimated the probabilities of future events, you may choose to take the predictions as irreproachable (and risk disaster), or do you may require some human verification (introducing a manual step into a system you were attempting to automate).
If a system must remain fully automated then a decision must be made on how to map each output to a act. Most AI produces outputs in the form of one or more probabilities and confidence values. If the output is that an event will occur with a certain confidence, what are you going to do? Assume it will happen if confidence is above a given threshold? Then what threshold? Alternatively, you may evaluate the costs of incorrect decisions and try to minimise risk, but correctly evaluating costs and risk is a significant project on its own.
If a system is allowed to include a some manual verification (a so called human-in-the-loop) then clearly anomalous decisions can be avoided, but there are new risks introduced. If a system is mostly accurate then a human verifying the output may become complacent and begin to trust the decisions of the machine implicitly, at which point the human verification becomes useless. Another risk is that human may be biased in their decision to agree or disagree with the machine, at which point any benefit of the intricate calculation of the system may be lost.
Whatever decision is made, you must also accept that the outcome will affect future behaviour. If a system is makes decisions that affect a person's life in a meaningful way, by setting insurance premiums for example, they may change their behaviour in an attempt to influence the system. This change in behaviour could be: benevolent, "My health insurance premium seems to be lower since I stopped smoking"; malevolent "I've noticed that for every salad I purchase my premium reduces by a few pence, so I've started ordering thousands of salads online and immediately refunding the purchase"; or irrelevant, "I seem to get offered lower premiums on Tuesdays, so never bother renewing on days beginning with a 'T'"? The unfortunate effect therefore is that if the introduction of a system causes a change in the events it was attempting to predict, then the data on which it was trained may immediately become obsolete.
There is a lot more to an AI project than the algorithm. A poorly chosen the problem may be completely insoluble, a poorly defined context may invite failure as the system is applied to situations for which it was not designed, and poor interpretation of the output may cause ethical or political problems down the line.
Please send questions or comments to andrew@aomartin.co.uk.