Skip to content

Latest commit

 

History

History
201 lines (147 loc) · 15.4 KB

Guidelines.md

File metadata and controls

201 lines (147 loc) · 15.4 KB
Raw

AI Policy Guidelines

The purpose of these guidelines is to assist organizations in effectively implementing and managing generative artificial intelligence (Generative AI). The document is structured into categories, each addressing a significant aspect of AI use.

Each topic within these categories is presented at three levels: Good, Better, and Best:

  • Good: Represents a foundation level that covers essential practices.
  • Better: Builds upon the good practices, offering a more optimized balance between resource allocation and advancement in AI practices.
  • Best: Represents the ideal practices, which may require substantial resources but can potentially yield high returns.

It's worth noting that the "Best" level isn't necessarily a goal for every organization. Rather, it serves as a reference point for what is possible given unlimited resources. The "Better" level, on the other hand, is often a more realistic target for many organizations, given its balance of costs and benefits.

Organizations may refer to these guidelines in light of their specific context, resources, and strategic objectives. The goal is to incorporate AI in a responsible and efficient manner that aligns with your organization's mission and values.

Table of Contents

  1. Ethical Use of AI
  2. Managing Risks Associated with AI Usage
  3. Risk Evaluation and Mitigation
  4. Oversight of AI Usage
  5. Dispute Resolution and AI Misuse
  6. AI and Biases
  7. Data Shared with AI
  8. Use of Authorized AI Tools
  9. AI Training and Development
  10. AI Experimentation
  11. Using Company-Provided AI Tools
  12. Access Control to AI Tools
  13. Controlling AI Tool Installation & Data Access
  14. Reporting New AI Uses
  15. Integration with Other Organizational Policies
  16. Continuous Review and Update of Policies
  17. Avoiding LLM Hallucinations
  18. Ensuring AI Accuracy

# Ethics and Risk Management

The ethical use of AI, especially Generative AI, requires organizations to ensure fairness, transparency, and respect for privacy. Additionally, the risks associated with AI usage, from inaccurate outputs to potential misuse, need to be carefully managed. Failing to do so can not only lead to operational issues but also damage an organization's reputation and legal standing.

1. Ethical Use of AI

Policy Level Description
Good Encourage employees to uphold ethical standards when using AI, including respecting fairness, transparency, and privacy.
Better Provide employees with detailed ethical guidelines for AI use, including case studies and potential scenarios.
Best Create an AI Ethics Committee to monitor AI use, provide ethical guidelines, handle dilemmas, and promote ongoing ethical education related to AI technologies.

2. Managing Risks Associated with AI Usage

Policy Level Description
Good Encourage employees to thoroughly review all AI outputs for accuracy, relevance, and potential issues, including any costs associated with unoptimized or excessive use. Promote a culture of consultation and peer review when employees are unsure.
Better Require employees to conduct detailed checks on AI outputs for significant or high-risk tasks. Implement AI usage guidelines and/or trainings that help avoid common pitfalls or exposure to sensitive information.
Best Establish a comprehensive framework for AI risk management, including procedures for testing, verification, and validation of AI outputs in high-risk areas. Regularly conduct AI risk training and audits. Develop a rapid response strategy to address any AI-related issues that arise.

3. Risk Evaluation and Mitigation

Policy Level Description
Good Encourage team leaders to regularly evaluate the risks associated with AI use in their departments.
Better Conduct organization-wide AI risk assessments on a semi-annual basis.
Best Establish a dedicated AI Risk Assessment Team responsible for ongoing evaluation and mitigation of AI risks. Implement AI-specific risk management strategies based on their findings.

4. Oversight of AI Usage

Policy Level Description
Good Encourage managers and team leads to have regular discussions with their teams about how they are using AI in their work, to understand and track usage patterns.
Better Implement a process for employees to document and report their AI usage to their managers or a designated person, such as the AI Ethics Officer. This could be in the form of quarterly reports.
Best Implement a monitoring system to track and report on AI usage across the organization, ensuring comprehensive oversight. This system should respect employee privacy by focusing on aggregate, anonymized data.

5. Dispute Resolution and AI Misuse

Policy Level Description
Good Encourage a culture of open communication where employees feel safe to raise concerns about AI misuse.
Better Implement a process for reporting and investigating potential AI misuse.
Best Establish a formal dispute resolution procedure for handling issues arising from AI use. The process should be transparent, fair, and should outline potential penalties for intentional misuse.

6. AI and Biases

Policy Level Description
Good Encourage employees to be aware of and account for potential AI biases in their work, such as those based on gender, race, age, or socioeconomic status.
Better Provide employees with training on how AI can reflect and perpetuate biases. For example, if an AI is trained on data that contains gender bias, it may make recommendations that unfairly favor one gender over another.
Best Create a dedicated team to monitor AI outputs for potential biases and develop an action plan for addressing any identified issues. This team should work proactively to minimize biases in AI inputs and processes, guided by a strong commitment to fairness and equality. For example, they might regularly test AI candidate recruitment systems with a diverse range of inputs to ensure fair treatment of all groups.

# Data Security and Privacy

As employees incorporate AI tools into their workflows, these tools often require access to significant amounts of information, some of which might be sensitive or confidential. Inadvertently granting an AI tool or add-on access could potentially expose an organization to data breaches or privacy violations. It's essential for organizations to be cognizant of the access they are granting to these tools and to establish rigorous guidelines and monitoring systems to protect their data.

7. Data Shared with AI

Policy Level Description
Good Encourage employees to think carefully about the data they share with LLMs or other third-party tools, considering privacy and security implications.
Better Provide explicit guidelines on types of data that should not be shared with AI, including private, sensitive, or confidential information.
Best Prohibit employees from disclosing any sensitive data, and list specific examples of things employees may not want to feed into AI tools like ChatGPT. Also, offer regular training and audits to ensure compliance.

8. Use of Authorized AI Tools

Policy Level Description
Good Encourage employees to use authorized AI tools with known privacy measures in place.
Better Require employees to use only specific, authorized LLMs with privacy measures in place, such as ChatGPT's private chat history & training.
Best Implement a whitelist of AI tools that are approved for use, with clear instructions on how each should be used to ensure privacy and security. Require employees to use only tools on this whitelist for work-related tasks, and conduct regular reviews and updates of the list.

# AI Tools and Training

Misusing AI tools can lead to serious issues, including privacy breaches and data leaks. Organizations should maintain oversight and control access to AI tool and invest in AI training and development.

9. AI Training and Development

Policy Level Description
Good Encourage employees to seek out training and development opportunities in AI technology.
Better Provide regular in-house training sessions and resources for AI technology.
Best Establish a comprehensive AI training and certification program for all employees. Require completion for anyone working directly with AI technologies.

10. AI Experimentation

Policy Level Description
Good Experiment with free AI tools such as ChatGPT, free image generation tools, and look at new AI features in your existing tech stack. Use tools like Canva and AppSheets.
Better Experiment with ChatGPT+, Midjourney, freemium browser plugins like Compose.ai, and look at new AI features in your existing tech stack. Utilize integrations/connecting up (Zapier) and experiment with APIs (whisper).
Best Experiment with OpenAI APIs and Github Co-pilot. Consider setting up experimental projects, like building a tech stack that includes an LLM on a cloud provider, or use transfer learning to make domain-specific bots (if you have enough training data).

11. Using Company-Provided AI Tools

Policy Level Description
Good Encourage employees to use company-provided AI accounts, such as a ChatGPT account, for any work-related AI tasks. Make it clear that these accounts are for business use only, and personal usage should be minimized or avoided entirely.
Better Require employees to use company-provided AI accounts for all work-related AI tasks. Establish guidelines around what constitutes appropriate use of these accounts and provide regular training to ensure employees understand these guidelines.
Best Establish a policy that all work-related AI tasks must be conducted using company-provided AI accounts. Implement monitoring and auditing mechanisms to review usage and enforce the policy. Offer regular training sessions and provide a channel for employees to ask questions or report potential misuse.

12. Access Control to AI Tools

Policy Level Description
Good Encourage employees to use only approved AI tools and discourage them from accessing unauthorized AI websites or services from their work devices.
Better Implement a policy that only allows employees to use approved AI tools for work-related tasks. Regularly review and update the list of approved tools.
Best Use network-level controls, such as firewalls or content filtering, to block access to unauthorized AI websites or services from work devices. Regularly review and update these controls to reflect changes in approved tools. Be transparent with employees about these controls and provide a clear rationale and guidelines for their use.

13. Controlling AI Tool Installation & Data Access

Policy Level Description
Good Implement a policy stating that all software installations, including AI extensions, must be approved by a manager or IT department. Encourage employees to check with their supervisor or IT before downloading any new software.
Better Provide a list of approved software and extensions that employees are permitted to install and use. Require IT approval for any software not on this list. Use permissions settings in your organization's software to prevent unapproved installations.
Best Implement a strict policy where only the IT department can install new software. Use centralized IT management tools to control software installations and updates on all company devices. Also, establish a whitelist of approved applications and blacklist unapproved or risky applications. Regularly educate employees on the risks of unauthorized software and the importance of data security.

14. Reporting New AI Uses

Policy Level Description
Good Encourage employees to share new AI uses with their manager or designated person/process.
Better Employees must report all significant uses of AI to their manager or designated person/process.
Best Employees must use the company's designated AI tracking system to document all significant uses of AI, including purpose, data inputs, and outcomes.

15. Integration with Other Organizational Policies

Policy Level Description
Good Encourage employees to consider how AI policies relate to other existing organizational policies.
Better Conduct a formal review to ensure alignment of AI policies with other organizational policies such as HR, IT, and data governance.
Best Integrate AI policies into the organization's overall policy framework, ensuring consistency and alignment across all areas. Regularly review and update these to avoid contradictions or conflicts.

16. Continuous Review and Update of Policies

Policy Level Description
Good Encourage regular team discussions to assess the relevance and effectiveness of AI policies.
Better Conduct formal reviews of AI policies at defined intervals (e.g., annually).
Best Establish a dedicated team or role for continuously monitoring advancements in AI, and updating AI policies to reflect changes in technology, regulations, and societal expectations.

# Special Topics

AI hallucinations (false statements made by AI) can lead to costly mistakes. Organizations should be aware of these issues and have strategies in place to handle them.

17. Avoiding LLM Hallucinations

Policy Level Description
Good Encourage employees to be vigilant for potential LLM "hallucinations" or false statements, and cross-check all AI-generated information.
Better Provide training to employees about common types of AI "hallucinations" and methods for identifying and mitigating them.
Best Implement a robust system of checks and balances for AI outputs including peer reviews, automated fact-checking, and manual validation. Include specific steps to address and learn from any identified hallucinations.

18. Ensuring AI Accuracy

Policy Level Description
Good Encourage employees to verify the accuracy of AI outputs, cross-referencing any factual claims with reliable sources.
Better Require employees to review all products of AI for accuracy. This includes manually cross-verifying all assertions, assumptions, etc.
Best Develop a rigorous verification system involving multiple employees and automated processes. This system should include feedback loops to improve AI accuracy over time. Provide regular accuracy reports to promote accountability and continual improvement.