目錄
On October 15, 2026, Anthropic announced that it had brought six members of the nonprofit research organization METR into the company, granting them system access comparable to that of its internal risk assessment team and allowing them to publish AI safety research independently. Known as "Embedded Evaluation," this arrangement aims to give third-party researchers direct access to model development and safety management processes, rather than requiring them to rely entirely on testing reports released by companies.
For the AI industry, this is a notable experiment in independent oversight. As models such as Claude become capable of performing long-running tasks, using external tools, and handling complex workflows, safety becomes increasingly difficult to assess through model benchmark scores alone. However, while METR can publish its research independently, it does not have unrestricted access to all of Anthropic's internal information. Some information may even have to be removed at the company's request, meaning the arrangement still falls short of a fully independent auditing system.
This development also matters for everyday workplace tools such as Notion AI and AI Agents. When AI can search corporate knowledge bases, create documents, modify databases, and even operate other services through connectors, model safety is only one part of the equation. Data processing policies, agent permissions, and workspace administration all affect the risks involved.
What Are Anthropic and METR Collaborating On? External Researchers Gain Access to Internal Evaluations
METR stands for Model Evaluation and Threat Research, a nonprofit organization that studies the capabilities, risks, and evaluation methods of frontier AI models. The two organizations have previously collaborated on reviews of Anthropic's model risk reports. The newly announced arrangement takes METR beyond relatively limited external reviews and brings its researchers closer to Anthropic's internal safety evaluation work.
According to Anthropic's official announcement on October 15, six METR members have received company-provided equipment, office space, and access to tools and systems broadly comparable to those available to its internal risk assessment team. Anthropic has also encouraged employees to treat these researchers as internal colleagues working on model risk, sharing necessary information and research context as long as restricted information is not involved.
How Does Embedded Evaluation Differ From Conventional AI Model Testing?
Traditional third-party AI evaluations typically begin by examining model outputs. Researchers design questions or test tasks, then observe whether a model follows instructions, produces dangerous content, or exhibits unusual behavior in different situations. Such tests are useful for comparing model performance under specific conditions. However, when questions involve training processes, internal monitoring, or deployment decisions, external researchers often have little choice but to draw inferences from limited information.
For example, a model might perform well on public safety tests but have previously bypassed monitoring systems or attempted unauthorized operations while carrying out tasks inside the company. If the company does not voluntarily disclose these incidents, external researchers will struggle to determine whether such failures have occurred. They will have even less insight into how the incidents were discovered, how they were addressed, and whether similar problems could happen again.
Embedded evaluation attempts to reduce this information gap. In addition to testing models, researchers may have opportunities to examine how safeguards are designed, whether monitoring systems function properly, and how companies handle known safety incidents. Evaluation can therefore extend into model development and deployment processes instead of remaining limited to success rates on individual tests.
However, having employee-like access does not mean researchers can view everything. Anthropic retains restrictions on certain categories of information, including customer data, legally privileged information, and some details of model architecture. Consequently, the questions METR can investigate will still depend on the terms of the collaboration and the information available to its researchers.
Anthropic Establishes a Three-Tier System for Third-Party AI Safety Evaluations
Anthropic also announced a three-tier external evaluation framework that it is developing. The framework aims to provide different levels of information access according to the scope of research and the maturity of each collaboration. The main differences between the three tiers concern how much internal information researchers can access and whether publishing their findings requires Anthropic's approval.
| Evaluation Tier | Scope of Information Access | Research Publication Rights | Current Collaboration Status |
| Tier 1 | Limited information provided for specific risk-related issues | Anthropic's approval is required before publication | SecureBio |
| Tier 2 | Access to more internal information than Tier 1 | No prior approval required, but specific information-removal restrictions apply | Collaboration tier under development |
| Tier 3 | Access comparable to that of the internal risk assessment team | Research can be published independently, subject to restrictions on sensitive information | METR |
METR is currently the evaluation organization publicly confirmed to have received Tier 3 access. Another nonprofit organization, SecureBio, is reviewing the chemical and biological weapons-related sections of Anthropic's August 2026 risk report under a Tier 1 arrangement. Anthropic is also collaborating with external teams, including Accenture, on evaluations. However, these partnerships do not mean that every collaborator has the same access rights as METR.
This tiered framework reflects a practical challenge facing AI safety evaluations. Companies need to provide external researchers with sufficient evidence while preventing the disclosure of customer data, trade secrets, or technical details that could be misused. The different tiers represent an attempt to find workable forms of collaboration that balance these competing needs.
Can METR Publish AI Safety Reports Independently? Anthropic Still Retains Information Restrictions
The independence of an external evaluation cannot be determined solely by whether researchers are company employees. It is also necessary to examine who determines the research scope, whether researchers can publicly express disagreements, and whether the company has the authority to prevent unfavorable findings from being published.
Under the new collaboration terms published by Anthropic, METR can independently release analyses of model risks, internal incidents, and safety measures. Anthropic no longer has a blanket right to veto the publication of research findings. In principle, even if METR criticizes the company's assessment of its risks, it does not need Anthropic's prior approval. This represents a meaningful step beyond traditional arrangements in which companies must authorize the publication of evaluation results.
Anthropic Cannot Veto Reports Entirely, but It Can Require Sensitive Information to Be Removed
Independent publication still comes with restrictions. Anthropic can require METR to remove information falling within predefined categories, such as customer data, legally privileged material, and certain confidential intellectual property. Anthropic also retains the final authority to determine which information falls within these restricted categories.
METR can publicly explain what information it was able to access, what restrictions affected its research, and whether it agrees with Anthropic's reasons for requesting removals. However, it cannot use these explanations to disclose the underlying restricted information directly.
This arrangement creates a potential problem. If evidence of a particular safety risk involves confidential company information, external researchers may reach independent conclusions without being able to publish all the supporting evidence. Readers may learn that researchers identified a problem but still be unable to examine all the evidence themselves.
On the other hand, requiring companies to disclose complete model architectures, customer records, or details of internal safeguards is not necessarily reasonable. Some information could increase the risk of attacks if leaked. A mature external evaluation system therefore needs clearly defined confidentiality boundaries, traceable information restrictions, and mechanisms for resolving disputes, rather than simply requiring everything to be made public.
Third-Party Research Is Still Different From Statutory Safety Audits
Anthropic has previously compared embedded evaluation to banking supervision. Large banks generally possess more comprehensive information about their internal risks than outside observers. Regulators may therefore place personnel inside financial institutions to continuously examine their risk management practices.
However, METR is not a government regulator, and this collaboration is not an on-site audit mandated by law. The researchers' access rights arise from an agreement between the two organizations. They have not gained statutory investigative or enforcement powers. Nor should METR's presence inside Anthropic be interpreted as evidence that Claude has passed any form of comprehensive safety certification.
The credibility of an evaluation also depends on whether research funding, collaborative relationships, and information access are influenced by the company being assessed. If an AI company can terminate a collaboration at any time, restrict access to specific evidence, or prevent researchers from investigating truly important safety incidents, the research may still suffer from inadequate coverage even if the final report remains unchanged.
On October 13, 2026, a Reuters investigation reported continuing disagreements between major AI companies and external evaluation organizations over publication rights, trade secret protection, and restrictions on collaboration across companies. Anthropic's proposed framework represents one attempt to address these issues. Whether it can maintain its independence over time will need to be established through subsequent research and evidence from its actual operation.
What Did METR's Latest Research Find? Anthropic's AI Monitoring Systems Still Have Shortcomings
Shortly after Anthropic announced its embedded evaluation framework, METR published another study on October 20, 2026. This research examined the section on "Misalignment in high-stakes settings" in Anthropic's August 2026 risk report.
The report helps illustrate how external researchers can challenge a company's safety assessments, but an important distinction in timing must first be clarified. METR explicitly stated that this research was conducted under its previous collaboration terms. During the research period, it did not have the embedded evaluation access announced on October 15. The report therefore cannot be considered the first evaluation produced under the new framework.
Under the previous terms, Anthropic had the right to veto publication and require content to be removed. However, METR stated that no content was ultimately removed and that its research conclusions were not changed during the publication review process. Only some wording and experimental methods were adjusted in response to feedback.
METR Agrees That Catastrophic Risks Are Currently Low, but Not Entirely With the Reasoning
Anthropic's original report concluded that, as of July 15, 2026, the risk of catastrophic harm caused by misalignment among the models covered by the report was low. Here, model misalignment primarily refers to AI behavior or objectives deviating from what was intended, potentially causing severe consequences under certain circumstances.
METR broadly agreed with the conclusion that current risks were low. However, it argued that this was primarily because the evaluated models still had limited capabilities, rather than because existing safety training or safeguards had necessarily proven sufficiently reliable.
In other words, concluding that current models have not yet demonstrated the capabilities needed to cause certain catastrophic outcomes is different from concluding that companies have established reliable safeguards capable of consistently preventing those outcomes. If model capabilities continue to improve, further empirical research will be needed to determine whether existing safeguards remain effective.
METR also argued that Anthropic's current risk evaluation methods are not yet sufficient to reliably identify all significant forms of model misalignment when risks genuinely increase. This criticism concerns the available evidence and evaluation methods. It does not mean METR has demonstrated that Claude will cause catastrophic incidents in ordinary use.
Passing Safety Tests Does Not Mean a Model Can Be Effectively Monitored in Real-World Environments
METR's concerns about monitoring systems are particularly noteworthy.
Many AI safety tests establish specific tasks and observe whether models attempt to conceal their behavior, evade monitoring, or violate established safety requirements. However, models deployed in real-world environments do not necessarily behave in the ways anticipated by test designers. In some cases, they may not even need to deliberately conceal their actions. Simply causing a monitoring system to mistake harmful behavior for part of a legitimate task could be enough to bypass the intended safeguards.
METR therefore concluded that some existing tests do not adequately represent a model's ability to circumvent real-world monitoring systems. Previous incidents have also weakened certain arguments about the effectiveness of safeguards. The research team recommended that Anthropic conduct more comprehensive end-to-end adversarial testing of its monitoring systems and expand its risk evaluations to include scenarios in which AI could influence internal corporate safety decisions.
In the same research report, METR also noted that some removed internal information was significant enough to affect researchers' assessments of model risks and monitoring capabilities. It therefore recommended that Anthropic release more evidence after applying appropriate protections.
This also demonstrates the practical value of external evaluations. Even when researchers do not overturn a company's original assessment of overall risk, they can identify gaps in the supporting evidence or failure scenarios that existing tests have not covered.
Why Do AI Agents Need More In-Depth Safety Evaluations? Risks Now Extend to Tools and Permissions
Some of the risks METR studies involve catastrophic scenarios that could arise from frontier AI models. These are not entirely the same as the data leaks or operational errors that businesses commonly encounter when using AI. However, both share a fundamental problem: a model performing normally in a test environment does not guarantee that it will continue behaving as expected once integrated into a complex workflow.
When AI was primarily used to generate text, organize documents, or answer questions, errors could usually be identified directly in its output. Today's AI Agents, however, can call tools, retrieve external data, modify records, and execute multistep tasks. Some operations may even be completed without requiring human confirmation at every step.
As a result, risk management must extend beyond model responses to the entire execution process.
Prompt Injection Can Cause AI Agents to Treat External Data as Operational Instructions
Prompt injection is a common security risk associated with agentic AI. It typically occurs when a model processes external data and mistakenly interprets untrusted content within that data as instructions it should follow.
Suppose an AI Agent is responsible for reading customer emails, organizing tasks, and synchronizing work progress to Notion. Under normal circumstances, email content is simply data to be analyzed. However, if an email contains instructions to search other customers' files, disregard existing rules, or forward information to an external address, the model must recognize that these instructions have not been legitimately authorized.
If the system cannot distinguish data from instructions, the agent may carry out actions that conflict with the user's intentions while completing an otherwise legitimate task.
This problem also demonstrates why instructing AI to follow safety rules through prompts alone does not provide comprehensive protection. External content can still influence model decisions. Therefore, the tools themselves must restrict which actions can be executed and require additional verification for high-risk operations.
Excessive AI Agent Permissions Can Turn Ordinary Errors Into Data Security Incidents
OWASP identifies "Excessive Agency" as an important risk in its classification of security vulnerabilities affecting large language model applications. The issue primarily involves excessive functionality, overly broad permissions, and too much autonomy.
For example, an agent whose only responsibility is organizing data might also be granted permission to delete documents, modify financial records, and send external emails. Even if it merely receives an ambiguous instruction, it could then cause consequences far beyond the scope of its intended task.
The problem does not necessarily arise from malicious behavior by the model. Sometimes the data source is incorrect, the model misunderstands the request, or external content has been manipulated. What allows an incident to escalate is a system that permits the agent to execute high-impact operations directly without appropriate confirmation procedures.
Safety evaluations should therefore examine which tools an agent can use, what permissions each tool provides, and which actions require human approval. For operations that modify production data, delete records, or transmit information externally, permission restrictions should ideally be enforced by the tools executing the actions or by downstream systems, rather than relying entirely on the model's own judgment.
How Does Anthropic's Third-Party Safety Evaluation Affect Notion AI Users?
Notion AI uses different model providers and inference services to deliver its functionality, including companies such as Anthropic and OpenAI. The safety governance practices of model providers are therefore relevant to Notion's AI services.
However, model safety, data privacy, and workspace permissions operate at different levels. Anthropic's acceptance of METR's evaluation can provide insight into how one of Notion's providers manages model risks. It does not mean METR has reviewed Notion's service architecture, nor does it mean METR can access the contents of Notion customers' workspaces.
For teams using Notion AI in practice, following model safety research is only part of the responsibility. They also need to understand which processors receive their data, how workspace content is retrieved, and whether agents possess permissions beyond what their tasks require.
Notion AI Does Not Use Customer Data to Train Models, but Data May Still Need to Be Transmitted and Processed
According to Notion's official AI security documentation, customer workspace data and personal information are not used for model training by default. Its AI subprocessors are also contractually prohibited from using customer data for model training without authorization. Notion separately offers a voluntary data-sharing program, whose eligibility and applicable terms should be reviewed independently.
However, when Notion AI needs to answer questions about a workspace, the system may still retrieve relevant content and send the information necessary to complete the request to a model service provider for processing. Therefore, "data is not used for model training" does not mean "data is never transmitted."
Data retention periods also vary by plan. According to Notion's currently published security documentation, default large language model providers for Enterprise workspaces operate under a zero-data-retention arrangement. For other plans, model providers may retain data for up to 30 days. These policies primarily concern data processing by model providers and should not be treated as equivalent to Notion's storage policies for workspaces, chat histories, or search indexes.
When customer contracts, personal data, or other sensitive information are involved, the actual arrangements must be confirmed against the features and plan being used, along with the applicable data processing terms. The model provider's safety statements alone are not sufficient.
How Do Permissions Differ Between Notion AI and Custom Agents?
Notion's official documentation makes a clear distinction between the permission mechanisms used by standard Notion AI and Custom Agents.
Standard Notion AI performs searches according to the current user's access permissions and, in principle, can only retrieve content that the user is already authorized to view. Custom Agents, by contrast, have their own independent resource permissions and can access specific pages, databases, and external applications authorized when they are created.
| Comparison | Standard Notion AI | Custom Agents |
| Permission basis | Current user's individual permissions | Resource permissions granted to the agent itself |
| Data access | Generally limited to content the user is authorized to view | Can access resources explicitly authorized for the agent |
| How it works | Primarily responds to user requests | Can execute automatically based on settings and triggers |
| Collaboration across users | Operates according to each user's permissions | Multiple users can share the same agent |
| Main risks | Data processing, inaccurate responses, and user permission management | Excessive agent permissions, indirect information access, and automated operation errors |
Custom Agents are designed for situations requiring access to information across departments. For example, project members might use a shared agent to check progress, or an agent might consolidate information from multiple databases.
However, this arrangement introduces a permission issue that can easily be overlooked. Notion's official documentation explicitly states that even if users do not have direct access to a particular page, they may still obtain information from it through the responses of a Custom Agent authorized to read that page, provided they have permission to use the agent. If the agent also has write permissions, users may even be able to modify resources they cannot directly edit.
This is a characteristic of Custom Agents' permission design. It does not mean that a new data breach has occurred at Notion, but it does require careful consideration when sharing agents.
Suppose an agent can read internal company quotations, budgets, and customer data, and all project members are allowed to use it. Even if the original databases have restricted sharing settings, the agent may create another pathway through which users can access that information.
Therefore, removing a member's access to the original page does not necessarily prevent them from obtaining the information indirectly. Administrators must also check which agents the member can use and whether those agents still have access to the relevant resources.
How Can You Establish an AI Risk Management Process in Notion? Start With Data Permissions and Activity Records
For freelancers, small teams, and businesses, building an AI safety evaluation system on the same scale as Anthropic's is generally unnecessary. A more practical approach is to first determine what data existing AI tools can access, which actions they can perform, and whether errors can be stopped or reversed.
These checks do not require disabling AI entirely or building a complex new management system. Organizing a list of services, permissions, and risk records within an existing Notion workspace can provide a foundation for subsequent periodic reviews.
First, Identify the Data AI Can Access and Review Provider Data Processing Policies
Workspaces often contain ordinary notes, publicly available materials, customer contracts, financial records, and unpublished plans. However, different types of data are not equally suitable for AI processing.
A useful first step is to classify which content can be processed directly by AI, which requires de-identification, and which cannot be processed by unauthorized services because of contractual obligations or internal policies. Once the data has been classified, teams can review which providers are actually used by each AI feature, how data is retained, and what terms apply.
This is particularly important when workflows involve integrations across multiple platforms. Data may pass through Notion, model service providers, connectors, and other automation platforms, so reviewing the privacy policy of just one provider is not sufficient.
Review Custom Agents and Connector Permissions to Prevent Excessive Access
After completing the data inventory, the next step is to review currently enabled agents and external connectors.
For Custom Agents, the Tools and Access settings can be used to determine which pages, databases, and external applications an agent is authorized to access. Teams should then review who can use the agent and whether its tasks genuinely require permission to read, modify, or transmit information externally.
If an agent only generates project summaries, it generally does not need database editing permissions. Similarly, if its work concerns a particular customer, granting access to the entire workspace should be avoided.
Teams should also check whether unused connectors, expired projects, or former employees still retain access. Depending on the features enabled, Notion Enterprise administrators can manage agent activity through Agent Directory, content search, and audit logs.
Simulate AI Agent Failure Scenarios to Verify That Errors Can Be Blocked and Reversed
Once permissions have been configured, teams can use test data to simulate common failure scenarios. Examples include an agent reading a page it should not access, modifying a database based on incorrect information, or performing unnecessary operations after being influenced by instructions embedded in an external document.
Testing should focus on actual execution results rather than simply examining how the agent describes its behavior afterward. If a system is supposed to obtain human confirmation before modifying important data, it must be verified that the agent stops before performing the operation, rather than requesting approval after the modification is complete.
For automated workflows with write or delete permissions, teams should also verify whether data can be restored through version history, backups, or other recovery mechanisms. They should establish who is responsible for disabling the agent, examining activity records, and managing the consequences if something goes wrong.
Create an AI Risk Register and Regularly Review Service and Agent Permissions
In addition to pre-deployment testing, AI tools that remain in use require periodic reviews. Models, providers, and platform capabilities may change, while permissions that were originally appropriate may become unnecessary after a project ends, personnel change, or new integration is added.
A simple "AI Risk Register" can be created in Notion to document the AI services currently in use, the data they can access, and their permission status.
| Field | Recommended Information to Record |
| AI Service Name | Notion AI, Custom Agent, or another AI tool |
| Model and Provider | Models currently in use and their associated service providers |
| Accessible Data | Pages, databases, customer data, or external sources |
| Operation Permissions | Read, create, modify, delete, or transmit externally |
| Data Sensitivity | Public, internal, confidential, or other classifications |
| External Safety Evaluations | Published third-party research and the scope of evaluations |
| Risks and Remediation Status | Identified issues, mitigation measures, and responsible personnel |
| Last Review Date | Most recent date permissions and settings were checked |
Individual workspaces can begin by identifying sensitive pages and removing unnecessary permissions. Teams that collaborate across multiple users should pay particular attention to indirect access enabled by shared agents. Workflows that automatically modify production data should also undergo failure scenario testing before being enabled.
Will Anthropic's Embedded Evaluation Become a New Standard for the AI Industry?
Anthropic's collaboration with METR offers a form of external oversight that goes deeper than traditional public model testing. Researchers can examine model development and risk management processes more closely, then form independent conclusions based on the evidence they obtain. This can help reduce the information gap between AI companies and the outside world.
However, whether the framework can remain effective over the long term depends on the research conducted under it. Outside observers need to understand which systems evaluators actually examined, which questions could not be verified because of access restrictions, and whether findings can be published without improper interference. Even when reports cannot disclose every internal detail, they should explain their methods, scope, and evidentiary limitations as fully as possible so readers can determine the circumstances to which their conclusions apply.
METR's review published on October 20 has already demonstrated the potential role of third-party research. Researchers do not necessarily have to overturn a company's overall risk assessment to identify shortcomings in testing methods, blind spots in monitoring systems, or internal information that warrants further disclosure. Whether the new embedded evaluation framework can provide more comprehensive evidence remains to be seen through studies conducted under the new terms.
For organizations purchasing AI services, future safety reports should also be evaluated according to whether they represent tests published by model providers themselves, evaluations commissioned from external organizations, or audits with formal regulatory authority. These three types of assessment offer different levels of assurance. The involvement of a third party does not automatically make them equivalent forms of safety review.
Conclusion: Third-Party AI Safety Evaluations Can Reduce Information Gaps, but Practical Use Still Requires Permission Management
By giving METR researchers access comparable to that of its internal risk assessment team, Anthropic has introduced a new collaboration model for third-party AI safety evaluations. Compared with assessments based solely on public tests and company reports, researchers now have opportunities to investigate more development processes, internal incidents, and safeguards. However, access to information and the publication of findings remain subject to restrictions, and the framework's independence will require continued scrutiny.
METR's latest research also highlights that low model risk today does not necessarily mean companies have established adequate safeguards for more capable models in the future. As AI capabilities and deployment methods continue to evolve, the effectiveness of existing monitoring approaches must be verified through new tests and empirical evidence.
For Notion AI users, whether a provider accepts external safety evaluations can be one factor when choosing a service. In day-to-day operations, however, it is even more important to understand how data is processed, what permissions agents have, and whether erroneous actions can be blocked or reversed. When using Custom Agents and cross-platform automation in particular, teams should review the sharing settings of both the original data and the agents themselves. This helps prevent information that was originally restricted from becoming indirectly accessible through agent permissions.
External research can help verify the safeguards implemented by model providers. Decisions about which workspace data to make accessible and which operations AI should be allowed to perform automatically must still be based on actual business requirements and the sensitivity of the data involved.
Common FAQs
METR is a nonprofit organization that studies the capabilities and risks of frontier AI models. Anthropic has granted six METR members access comparable to that of its internal risk assessment team so they can examine model risks, safeguards, and internal safety management processes.
Conventional external testing primarily examines model behavior through its outputs. Embedded evaluation allows researchers to go further by reviewing some internal incidents, monitoring mechanisms, and development processes, helping them verify the evidence behind safety reports.
Under the new terms announced in October 2026, METR can generally publish research independently. However, restrictions still apply to customer data, legally privileged information, and certain details of model architecture. Anthropic can require specific sensitive content to be removed.
No. METR stated that the report was completed under its previous collaboration terms and that its researchers did not have the embedded access announced on October 15 while conducting the study. The two developments should therefore be understood separately.
METR broadly agrees that certain catastrophic misalignment risks are currently low. However, it attributes this primarily to the models' limited capabilities and argues that existing monitoring and risk testing are insufficient to demonstrate that more capable future models can still be reliably controlled.
No. This is a voluntary collaboration between Anthropic and third-party researchers, not a statutory government audit. It does not constitute comprehensive safety certification for Claude or other products using Anthropic's models.
No. When answering questions, Notion AI may still retrieve workspace content and send necessary information to model service providers for processing. Whether data is used for training, how it is transmitted, and how long it is retained must be assessed separately under the applicable policies.
Standard Notion AI generally operates according to the current user's page access permissions. Custom Agents have independently authorized resource permissions. Users may therefore obtain information indirectly through a shared agent even when they cannot access the original content directly.
Identify sensitive data, minimize agent and connector permissions, require human confirmation for deletions, external transmissions, or significant modifications, and use test data to simulate prompt injection, incorrect writes, and recovery procedures.
They should determine what evidence evaluators could access, which models and scenarios were covered, whether publication restrictions applied, and what issues remain unresolved. A report should not be considered a comprehensive safety guarantee simply because a third party conducted the evaluation.