Principles for responsible AI product development
At Hyperact, we define Artificial Intelligence (AI) as a broad spectrum of technical solutions where the software system’s behaviour is not explicitly programmed by engineers. This encompasses traditional statistical methods, deep neural networks, and all related tools and methodologies.
AI is advancing at an unprecedented pace, reshaping industries and human-technology interactions. However, these advancements introduce significant ethical, security, and societal challenges that require proactive management. To address these challenges, we have established AI principles ensuring our solutions are:
- Fair – mitigating biases
- Robust – consistently accurate and safe
- Explainable – transparent in decision-making
- Secure – protecting users’ data and privacy
Our AI principles are not just guidelines; they represent our commitment to responsible AI development and deployment. They also serve as a reference for organisations adopting AI.
We encourage you to critically assess your AI work and how our principles apply. The quality and ethical alignment of AI are heavily influenced by broad organisational and societal factors. It is critical that both technical and non-technical stakeholders understand their contributions and the impact of AI solutions.
With that in mind, let’s look at where AI is making an impact - and where the risks lie.
Industry impact and risk overview
Research from McKinsey suggests that AI could add $2-4 trillion annually across multiple use cases such as marketing, sales, manufacturing and pricing. But this progress also introduces risks to both consumers and businesses, some of those risks include:
- Systemic bias and discrimination: Historical and structural biases embedded in AI systems, which can unintentionally reinforce societal inequalities.
- Discrimination in AI decision making: The unfair treatment of users in real-time decision making through AI interactions, such as chatbots, virtual assistants, and automated customer service systems.
- Security breaches: Novel methods for unauthorised access and exploitation.
- Intellectual property and copyright infringement: Risks related to the misuse of proprietary content.
- Disinformation: The potential spread of false or misleading information.
- Data manipulation: The misuse or mishandling of personal and sensitive information.
- Deep fakes: The creation and dissemination of deceptive multimedia content.
- Regulatory violations: Non-compliance with regulations such as GDPR and the EU AI Act.
- Mass surveillance: The risk of excessive data collection infringing on privacy rights.
🤝 Purpose: Ensuring AI is the right tool for the job
AI is not always the best solution. Choose it only when it adds value and avoids unnecessary complexity. The decision to incorporate AI into a system should be intentional, ensuring it directly addresses user needs, creates value, and does not introduce unnecessary complexity. Before implementing AI, organisations should critically evaluate whether AI is the best approach or if a simpler, more deterministic solution would suffice.
Your responsibilities:
- User-centered decision-making: AI should only be used when it demonstrably enhances user experiences, solves a real problem, and aligns with business goals.
- Problem-solution fit: Ensure AI is selected based on its ability to improve efficiency, decision-making, or automation rather than as a trend-driven choice.
- Simplicity first: Consider whether traditional, rules-based, or statistical models could achieve the desired outcome before opting for complex AI models.
- Risk-benefit analysis: Assess whether the risks associated with AI such as ethical concerns, security vulnerabilities, and explainability limitations are justified by the benefits it provides.
- Continuous validation: Periodically reassess AI deployments to determine whether they continue to add value or if alternative approaches would be more effective.
How to measure:
- Value assessment: Compare performance metrics of AI-driven solutions with non-AI alternatives to justify its usage.
- User feedback: Regularly collect insights from users to ensure AI is meeting their needs and not over-complicating workflows.
- Efficiency gains: Track AI-driven improvements in accuracy, automation, or decision-making versus traditional methods.
- Governance reviews: Conduct periodic reviews to assess whether AI remains the best tool for the problem being solved.
⚖️ Fairness: Ensuring equitable AI solutions
Users demand AI solutions that are equitable to all, ensuring no individual or group is subject to systematic bias.
Your responsibilities:
- Team diversity: You should strive to ensure that teams contributing to or building AI solutions are diverse groups of people. You should also invest in AI training for all contributors (MIT, IBM and DataCamp all offer great material) - ensuring a common baseline of education, this is particularly important for those working with training data or prompts.
- Definition of fairness: It is important that you acknowledge different domains require different fairness criteria, therefore you should define a clear definition of what fairness means within the context you are operating within and its tradeoffs (equity based on context, equal treatment, equal opportunity, etc).
- Data representation: Ensure the data used to train or prompt your model is representative of all relevant demographics and has undergone thorough validation which includes all data acquisition and labelling sources.
- Equity in outputs: Verify that a wide variety of model inputs and outputs remain equitable, especially if the training data is unavailable or "closed source”.
How to measure:
- Automated representation balance: Disparity ratios and demographic based performance (accuracy, precision, recall) comparisons when evaluating equality across diverse groups within training, context and output data.
- Human-led, diverse, stakeholder and expert feedback: Gather subjective assessments from a diverse group of stakeholders and experts to evaluate trust in the response and ensure it is perceived as unbiased.
🦾 Robustness: Consistently delivering relevant content
Users need AI systems that consistently deliver relevant content while avoiding toxic, incorrect, or inappropriate outputs. Results should be repeatable and consistent for the same input.
Your responsibilities:
- Prompt engineering: Craft prompts carefully to guide the AI effectively.
- Testing and validation: Establish concrete testing and validation methodologies to continuously monitor and constrain the AI's responses.
- Model management & ops: You should have a clear understanding on the types of models you have in development and production. Details such as: their purpose, training data & risk ratings, architecture, inputs/outputs and what actions they take.
How to measure:
- Automated repeatability tests: Measuring the semantic differences in responses given identical input and measuring this through stability metrics. Traditional metrics such as precision, recall, F1 scores and perplexity will help measure overall performance. Content specific metrics such as toxicity classification scores, NSFW content detection, and fact checking methods will ensure content is generally safe. Conducting adversarial testing, or stress testing the system against unexpected or dangerous inputs can help identify exploitation risks and quantify adversarial robustness scores.
- Human led rubrics and surveys: Measuring AI output against human response expectations can help understand overall output quality. Conducting manual exploratory testing of model performance with human experts can also identify nuanced quality risks.
🔍 Explainability: Ensuring AI decision-making is explainable
Users need decisions made by AI to be clear, understandable, and easily explained to those affected by them.
Your responsibilities:
- Traceability: Guarantee that all decisions taken by a model are traceable and explainable, both historically and during the user journey.
- Oversight: Set up robust oversight mechanisms including real-time monitoring, clear escalation protocols, and accessible interfaces that allow for human intervention whenever necessary.
- Model selection: Deploy AI models that are carefully selected, ideally from reputable sources that provide detailed information on their design, data sources, methodologies, and deployment processes. If you require a very high degree of explainability, you should consider this in your model selection analysis, understanding that some models are more explainable than others are key (e.g deep neural networks vs decision trees).
How to measure:
- Traditional logging and observability patterns: Document and keep track of decisions made by your AI solutions. When an AI is making a decision, you should measure confidence scores and feature attributions for those decisions, along with the outcome - as well as measuring the number of decisions with these measures. You should also measure the length of time between human reviews of AI decisions and your human intervention rate.
- Human in the Loop: Initially, observability data can be reviewed and validated by human experts before actions are committed. This allows for double-checking the model’s work and intervening in real-time if a decision appears incorrect. Over time, you should look to build automated ways of reviewing and auditing a model's response asynchronously.
🔒 Privacy & Security: Protecting sensitive information
AI models must resist attacks that extract sensitive data and avoid training on personally identifiable information (PII).
Your responsibilities:
- Data protection: Implement proper authentication, authorisation, and segregation of user data in AI deployments.
- PII management: Ensure that any PII is anonymised or redacted during both offline and online training.
- Data controls: Implement proper data redaction and anonymisation for model training. Ensuring also that all data flows into and out of the model are properly documented and understood. You should never perform one-shot or few-shot prompts using non-redacted PII outside, unless the data subject made the request themselves.
- AI management systems: Consider your compliance with ISO/IEC 42001 - which provides organisations a framework to ensure responsible, traceable, transparent, and reliable AI.
How to measure:
- Data anonymity: Standard data best practices also apply across all AI datasets. Due to the nature and scale of AI solutions, you’re more likely to be using a third party than not, so it is critical you follow your organisation's best practices and governance/risk assessments when working with third parties on AI solutions.
📜 Compliance: Ensuring legal & ethical AI
Users expect AI systems to be compliant with the law and regulation set out to protect them and their rights, therefore it is critical you are aware of, and understand the requirements and implications of laws and regulations surrounding AI solutions. In many cases, these regulations and laws can be industry specific and highly varied between governing countries / locations.
Your responsibilities:
- Legal awareness: Understand and implement the requirements of relevant laws and regulations, which can vary significantly by industry and jurisdiction.
This is a non-exhaustive list of some of the most critical regulations:
🌳 Societal & environmental impact: Fostering positive change
Users and stakeholders expect organisations adopting and shipping AI to account for their broader societal and environmental impacts. Ensuring that advancements in automation, data processing, and machine learning do not come at the expense of workforce stability or ecological sustainability.
Your responsibilities:
- Environmental considerations: Before implementing an AI solution, consider the smallest possible solution that will yield acceptable results - very large AI models impact the environment significantly, and while powerful they may be overkill for your needs.
- Resource optimisation: Deploy into carbon neutral data centers where possible, schedule AI training during off-peak periods, and make sure you’re optimising your machines for CPU/GPU usage.
- Human oversight & critical thinking: Encourage employees to question AI outputs and ensure that training and guidelines are well understood on how and when human oversight is mandatory, most importantly applied to decisions that impact employment rights, personal rights, healthcare, or legal scenarios at a minimum.
- AI literacy: It is key that you provide on-going training, documentation, and transparency to enable your teams and the organisations you work with to understand the relative risks, concepts, ethical considerations and capabilities of your AI solutions.
How to measure:
- Carbon & resource metrics: Energy Consumption (kWh), carbon offsets, and CO2 emissions tied to training and inference.
- Workplace impact: Monitor internal metrics on team retention, new role creation and/or displacement attributed to AI advancement. Assess the number of your employees completing your AI training, and governance programmes.
🔄 Feedback: Continuous monitoring and evolution
Users need AI that remains relevant and safe throughout the lifetime of the solution.
Your responsibilities:
- Governance mechanisms: Establish a core AI governance group responsible for periodically auditing AI solutions, covering areas such as data labeling, training data, model deployment, testing, and team composition.
- Key metrics: Teams should be required to publish real-time core health metrics for their deployed AI solutions, reporting key indicators such as accuracy, disparity ratios, and demographic-based performance metrics.
- Refinement: A clear process should be documented that requires teams to respond in a timely manner when AI begins to drift from an accepted baseline.
- Evolution: Teams should be encouraged to continually assess their user-base, fairness definitions, data assumptions and their governance processes in response to real world feedback, and observations.
What’s next
Adopting these AI principles ensures responsible and ethical development, benefiting both users and the broader community. As you embark on your AI development journey, remember to ask yourself:
- Are my AI solutions fair and equitable to all users?
- Are my AI systems robust and consistently delivering relevant content?
- Are the decisions made by my AI models transparent and explainable?
- Are my AI models secure and protecting sensitive information?
- Is human oversight available for high-stakes decisions?
- Are my AI systems transparent in their decision-making processes?
Throughout this piece, we’ve explored the challenges and opportunities of getting real value from AI. Our recent panel event reinforced many of these insights, featuring leaders from Peak AI, Neural River, Lloyds Banking Group, Versori and AutoTrader If you’d like to hear their perspectives, check out our event write-up.
In the next instalment of this AI series, we’ll explore different types of AI models, the risks and challenges they present, and practical testing and assurance methods you can apply throughout your AI development lifecycle.
