Should DataOps Engineers Be Made Accountable for AI Bias?

You don’t need machine learning analysis to know that AI’s impact on the business landscape is soaring.

Fortune Business Insights’ “Artificial Intelligence Market, 2020-2027” data suggests that the global AI market will reach USD266.92 billion by 2027. That’s on a spectacular CAGR of 33%, despite the current cautious spending climate.

But unlike other technologies, AI learns, evolves, and aims to be much more intelligent than its original self after years of ingesting data. This means future AI models may take over essential decisions in the future, relegating more of the complex and ambiguous ones for humans to debate and argue.

This means AI bias, which already affects society to some degree today as highlighted in this CDOTrends article, becomes a significant concern.

So who is accountable when an AI model makes a biased decision? Is it the data owners, the AI algorithm developers, the data science team, the data scientist, or the chief data officer? Recently, the finger seems to also point at the DataOps Engineer.

Workflows orchestrators

The role of the DataOps Engineer is relatively new. In fact, across the Asia Pacific, companies are only starting to understand the job scope.

The best way to visualize the work of a DataOps Engineer is to see DataOps as an assembly line. At its essence, DataOps is the science of building pipelines that take raw data and make it ready for analysis (or, in this case, for machine learning crunching).

The modern DataOps Engineer should not be confused with the traditional data engineer. The latter helped to prepare and transform data for data science use. DataOps Engineers design the entire data workflow so that data engineers and scientists can rapidly get insights with the fewest errors. In short, DataOps Engineers own the DataOps assembly line.

That sounds exciting until the ethics police come knocking on the door.

The new data fall guy

So why should the DataOps Engineer be responsible for AI bias?

To be fair, AI outcomes are a shared responsibility. Every step of the development of an AI model has people who should be responsible, from the data it is trained on to the algorithm that is chosen and the reinforcement learning done by domain experts.

In practice, no one admits responsibility. There is no U.N. Committee that dictates how ethical AI should run. Military and political interests can also impact ethical AI development. Only recently are governments waking up to the dangers of AI bias running amok.

But if there is someone that should be responsible, it should be the DataOps Engineer. He, she, or they are responsible for the data pipelines and the general workflow. While they do not work with data itself (unlike a data engineer, architect or scientist), they offer guidance and design support, code reviews, and new processes, argues this TechTarget article.

So, if they are responsible for creating data products that AI uses, it stands to reason that they will have to know if that data is biased.

“Equity as code” movement

However, the larger reason why DataOps Engineers should take on the AI bias accountability is that they have the means to root it out.

Enter equity as code. In DevOps, new code is automatically tested before deployment. If you extend the DevOps concept to DataOps (similar but not the same), DataOps engineers can add anti-bias controls, and metrics can be applied to new AI models.

Unlike traditional software development, AI models evolve and may “drift” as the algorithms are applied to new data. DataOps engineers can be continuously monitored for bias and other quality issues — a practice today called ModelOps.

A Forbes contributed article called this “equity as code” because it ensures equity is part of the testing process. DataOps engineers can extend “equity as code” by adding equity controls on AI algorithms or as part of the ModelOps processes.

Creating AI accountability

Ethical or unbiased AI development revolves around three related factors: functional performance, the data the AI model uses, and how the AI model itself is used.

The first two areas are well studied. We’ve also gotten better at making our AI technologies create the right outcomes. Whether these outcomes are accurate, fair and legal is where the concerns lie.

We’ve also gotten better at preparing data for AI use — the second factor. DataOps Engineers will bring a new level by orchestrating data workflows on the fly. Again, we lack the anti-bias safeguards and frameworks for these engineers to work with, but these are being debated, hashed out and formed among vendors and other stakeholders.

However, we’re not doing that well in addressing the third factor. It is challenging to prevent unintended biases and ensure that the AI models do not discriminate.
This is where many of the bias failure stories usually begin — from Microsoft’s famous and now defunct AI Chatbot spewing expletives to Amazon’s secret yet biased AI hiring tool that discriminated by using past resume data.

Admittedly, this is a difficult area to solve. It is one reason why many AI successes are framed within narrow or specific use cases. Operationalizing AI and broadening the AI model for other use cases can cause model drift and unearth bias.

But it is also an area where DataOps engineers are best prepared for. Armed with an expanding arsenal of new tools, industry knowledge and new frameworks, they can address an area of AI that many are unwilling to tread. But, of course, whether they will admit that AI accountability is their responsibility is a different matter.

Winston Thomas is the editor-in-chief of CDOTrends and DigitalWorkforceTrends. He’s a singularity believer, a blockchain enthusiast, and believes we already live in a metaverse. You can reach him at [email protected].

Image credit: iStockphoto/Wanlee Prachyapanaprai