Suppose you’re like many business leaders today, artificial intelligence (AI) has become a front-of-mind priority, specifically, how your organization can best harness AI’s power to transform your organization with digital age efficiency and boost bottom lines.
Today, AI technology is becoming a competitive need-to-have across industries—with 94% of business leaders agreeing that AI is critical to success over the next five years, according to the latest edition of a Deloitte study.
However, AI adoption is far from a guaranteed success. AI adoption comes with numerous challenges–finding the right technology vendors and in-house talent, aligning company culture, identifying appropriate use cases, integrating the technology with current processes, preparing to manage the inevitable changes, and governing the many potential risks associated with AI deployments.
Organizations looking to integrate and benefit from AI technology successfully need a structural body to provide organizational guidance and governance along the adoption journey. This body is a Center of Excellence (COE), which will drive and champion AI innovation across your organization. This article will discuss why an AI COE is critical for success, the best practices for setting up your COE, how to empower your COE to govern AI deployments effectively and share examples of COEs operating successfully today.
Why your organization needs a COE
Unfortunately, many organizations view AI technology as an a la carte, plug-and-play solution that delivers a fast, prodigious return. They will invest heavily in data infrastructure and specialized tools only to end with disappointed stakeholders when the big wins don’t materialize overnight.
Gartner statistics through 2022 paint a bleak but realistic picture–85% of attempted AI projects fail to deliver valuable outcomes. These failures occur for many reasons, starting with an overall scarcity of AI talent and the many cultural and organizational barriers that can impede the successful integration of AI into business practices and processes.
AI adoption also comes with many risks that can derail a deployment and possibly sink your entire organization if not properly governed. AI models use vast volumes of training data to identify patterns and make statistical predictions, and if the data contains biases, these inaccuracies can result in unfair or discriminatory decisions. Also, when handling large volumes of data, AI practitioners must be careful not to violate privacy laws and ensure data is safe from internal and external threats. These risks can result in costly consequences such as penalties, fines, jail time, or reputational damage.
Successful AI adoption starts with addressing relevant business problems. The path to success requires many internal and external action steps, overseen by a centralized group of diverse experts devoted to applying the technology for maximum business value while minimizing potential risks. This dedicated group is your COE, and without this supportive team leading the initiative, your business’s dream of AI adoption may unfortunately remain just that–a dream.
Another emerging reason for developing an AI COE is proposed regulations, such as one pending European law that would require a human component in the AI development loop. The proposed EU AI law would mandate establishing human controls to oversee and manage AI system behavior.
Responsibilities of your COE
What are the core responsibilities your COE will handle?
The following vital duties are critical to successful AI adoption and should fall to the experts within your AI COE:
- Determine your organization’s level of ambition and create a company vision for AI
- Develop an internal network of advocates and champions across the organization to solidify everyone’s connection to the WHY (why AI is necessary for the company, why it should be important to them personally, and how they will fit into the new AI culture)
- Work with organizational stakeholders to generate a “pipeline” of business-driven AI use cases and a roadmap for these use cases.
- Ensure responsible design, development, and deployment of AI projects by creating governance policies, procedures, and processes.
- Develop a rubric to prioritize AI use cases based on criteria such as impact, feasibility, and risks.
- Oversee the selection of the proper infrastructure platforms and needed tools, which should be user-friendly, automated, scalable, high-performing, and cost-efficient.
- Be the bridge of communication and collaboration with the business minds and stakeholders to ensure their support and help the projects deliver maximum business value.
- Track metrics and share early success stories with a priority on use cases delivering relevant results such as increased revenue and lowered costs
- Highlight the early wins to boost organization-wide support and maintain the appetite for future AI projects.
- Institute and oversee organizational AI training programs to develop a network of trained specialists and boost general AI knowledge across all departments.
- Build an ecosystem of external resources, including academics, vendors, and other AI adopters, to bring external insights and best practices into the organization.
Best practices for building your COE
How should you build your AI COE? What positions will you need to fill? How should it be structured?
There is no handbook with a step-by-step formula for constructing your AI COE. There is also no accepted best COE design, but successful teams seek a balance of centralization and flexibility.
Here are our recommended best practices for building your AI COE:
Assess AI maturity level
To get where you want to go, you must start by assessing where you are. What is your current readiness level to begin AI adoption? What resources, talent, tools, and knowledge does your organization possess? What will your team need to acquire? It can be helpful to familiarize yourself with Gartner’s AI Maturity Model, which explains the five levels of AI maturity–Aware, Active, Operational, Systemic, and Transformational.
Think centralized structure
Remember, AI talent is hard to find, so the best results often come when keeping your COE core team small with a tight-knit, centralized core of experts. One VP of Technology Innovation at a large insurance firm explains the importance of centralization when creating your COE, “Before getting power users and democratizing the consumption of AI, it’s critical that guardrails like cybersecurity are foolproof, and you need a central body to take the guardrails to the factory model level. Creating the initial governance structure, down to tracking the benefits, must be set up as a framework and industrialized, and for this, the role of your COE is critical.”
Many teams realize the benefit of a hub and spoke COE model where a centralized group embeds staff in business sectors heavy with AI deployment. These staff can familiarize themselves with and report on unit issues while building critical relationships with business function executives. Be sure to identify which activities the hub should handle (data governance, team building, vendor management, and training) and which activities best delegated to the spokes (adoption-related issues, workflow redesign, managing performance, and tracking metrics) that are closer to the AI system users.
Become agile and iterative
AI projects perform best in agile mode—run many short tests, get rapid results, and communicate often with stakeholders to keep motivation and support up. Testing often and failing fast is the mantra for teams looking to learn and adjust models quickly while keeping costs down.
In the rapidly shifting world of AI technology, remaining agile is crucial. Max Mayberry, Senior Software Developer at the Bunch and co-founder of Ai-productreviews.com explains, “One of our biggest challenges is the ever-changing technology landscape. We have adopted agile methodologies that enable us to rapidly iterate and seamlessly integrate new technologies to stay ahead of the curve. This agile approach allows us within the AI COE to remain at the forefront of AI, providing cutting-edge solutions to our customers.”
Align with business functions
When stakeholders witness concrete, viable results they care about, support for AI adoption undeniably grows. Will production teams support an AI use case capable of automating a mundane manual process and speeding up their job while freeing them up for a more engaging, value-added role? Will executives back a use case capable of speeding up time to market and increasing revenue while lowering production costs?
Most likely, they will—and your COE must serve as the critical link ensuring use cases connect with business functions and their relevant challenges. AI must become ingrained in your organization’s business functions, implemented to solve prioritized business problems, and not just a one-off, side-dish experiment. Design your COE to align and collaborate with business and operations teams, developing a steady pipeline of use cases where AI can deliver sufficient business value.
Build strong teams on interdisciplinary collaboration
Your COE will be the engine of your AI initiatives, assuming a leadership role with enthusiastic technology champions trusted as go-betweens communicating effectively with the tech and business camps. COEs are only as strong as the people within—so choose your talent and leaders wisely. Also, remember the significant talent gap in AI today—so when you acquire top talent for your COE, be sure to retain them.
Successful AI COEs build cross-functional teams of experts from across the organization with diverse perspectives and skills. This eclectic compilation of professionals–AI/data scientists, machine learning engineers, project managers, operations specialists, business leaders, and risk management professionals–working together ensures your COE will align project initiatives with organizational priorities.
What roles are essential to fill in your AI COE?
Titles and duties can vary by organization, but here are some primary roles you should consider when staffing your COE:
Executive Sponsor – A Chief AI Officer or VP of AI can fill this vital leadership role. They must be a decision-maker who understands the business domain, the business opportunities enabled by AI, and the risks of AI adoption.
Domain Expert – This essential role is the connecting link—providing the industry knowledge that drives the business requirements for the AI solution. The domain expert will also liaise with business stakeholders throughout the project.
Program Manager – This leadership role is needed to oversee the entire project, managing all other roles to ensure smooth function and guide progress.
Change Management Expert – Change and disruption are inevitable as organizations infuse business functions with AI systems. This role is required to oversee the management of these changes–attempting to keep the disruption to a minimum and helping make the adoption process as seamless as possible. Communication, training, and support during user testing are areas where a change management expert can help assist adoption with stakeholders and end-users.
AI Strategist – This position plays a critical go-between, connecting role by understanding one or more business domains while also having a working knowledge of AI technology. As an informed intermediary, this role coordinates with the executive team and external stakeholders to ensure the organization has the necessary infrastructure, resources, and talent to deliver successful AI project outcomes.
Ethics leader or council – Practicing responsible, ethically-driven AI development is paramount today, so it’s worth assigning an individual or group to monitor all AI-related practices, ensuring they stay true to the highest ethical standards. Their duties include staying current on privacy laws and compliance regulations governing personal data handling, storage, and usage.
Putting the COE team members in place is just the first step. Ensuring they collaborate and work as a unit despite different backgrounds is just as important. Whether your AI project addresses the proper use cases in the appropriate business functions and delivers the desired business value, making your adoption successful is contingent on collaboration—especially with the business stakeholders.
Orchestrating collaboration across departments must be a priority when forming your COE team. “We have invested in building a culture of collaboration and inclusion within the AI COE. Creating an environment that welcomes different perspectives has been shown to boost creativity and solve problems, a key factor in our success,” explains Mayfield.
Frameworks and Tools to Empower Your COE
Your COE will be responsible for overseeing the governance of your AI projects. Frameworks like Microsoft’s Responsible AI Standard, AI Ethics Impact Group, AI Ethics Framework for the Intelligence Community, and the NIST AI RFM exist to help COEs provide rules, organization, and guidelines around AI development and use.
The market of AI governance software tools has grown with the expanding technology. This market can be complex and confusing for those seeking governance help for their COE team. Below, we break down the four major categories of AI governance tools, which include:
Data governance platforms- These tools help manage your AI training data, ensuring its quality, integrity, and compliance with privacy laws. Leading brands include Cloudera, Devron, and Databricks.
Machine learning operations (MLOps) platforms- This category of tools and infrastructure helps your team manage the entire machine learning lifecycle. DataRobot and Dataiku are some key vendors in this space.
MLOps tools focus on addressing issues like data biases or ensuring privacy law compliance within individual components of the ML process, such as development, monitoring, or deployment. Top brands include WHYLABS and Datatron.
Large language model operations (LLMOps) tools manage the entire lifecycle of LLM-powered applications by monitoring models for bias or any potentially unethical practices.
AI Portfolio and Project Management software — Shibumi’s AI Value Accelerator is an example of software providing the governance capabilities needed to ensure all AI projects are done ethically and managed responsibly. This product leverages stages and gates to move AI projects through a lifecycle. AI COEs can set up automated notifications and role-based access to ensure that the right people engage at the right time to evaluate the risks and value associated with AI projects and ensure successful deployments.
Selecting the best AI governance tools can seem overwhelming for those starting to build their COE. Here are some general steps to follow when evaluating governance architecture and tools for your COE.
First, identify your organization’s needs based on your initiative and scale. Second, understand how each tool can address specific governance issues like detecting or preventing data bias and unethical practices, help organizations document the rationale behind AI models’ decisions, and ensure privacy regulation compliance. Third, grade the tools on their ease of use and ability to integrate with your current infrastructure, data sources, and storage solutions. Finally, compare brands on training offered, customer support resources, data security, and the cost to fit your long-term budget.
Looking forward
The day is coming when AI adoption becomes the differentiator between business haves and have nots. As organizations look to remain competitive by transforming business functions with AI deployment, the role of the COE will gain prominence as the critical driver of maintainable AI adoption and growth.
Many companies today are already seeing COE success. IBM’s AI COE is pioneering in AI research and development. Microsoft’s AI and Research Group is dedicated to advancing research and development in conversational AI and machine learning (ML), emphasizing integrating both into products and services. Salesforce’s Einstein AI is focused on advancements in customer relationship management (CRM) solutions to help businesses make more intelligent decisions around improving the customer experience.
COEs aren’t just for the big players—in fact, taking the time to build and develop a COE to support and guide AI project success is even more critical for those with fewer resources. The cost of establishing a COE is far less than attempting AI adoption without a centralized guiding structure to help support responsible development and use of the technology while controlling the risks.
If your organization is serious about AI adoption today, creating an AI COE is your first step toward success—the one critical step no organization can overlook, regardless of size or resources.