Skip to content
Safety Tech Review
Menu

Buying and piloting

40 questions to ask AI safety vendors: an RFP checklist

A 40-question RFP checklist for AI safety vendors, grouped by fit, accuracy, deployment, privacy, AI governance, security, workflow and commercial terms.

By · Updated · 11 min read · 14 sources

Reviewed by LIPAI WANG, Editor, on against the cited sources. How we review

This checklist details 40 questions to ask vendors of AI safety technology, like video analytics that detect unsafe acts and conditions. The list spans eight areas: fit, detection performance, deployment, data protection, AI governance, security, workflow and commercial terms. Each question highlights what a good answer looks like and the warning signs to watch for. Use it to build a request for proposal (RFP) or a shorter request for information. It grounds your comparison in evidence rather than presentation.

These questions assume a buyer in the US, UK or EU evaluating AI video analytics. Most of them also apply to proximity warning systems, wearables and EHS software with AI features.

How should you use this checklist?

Before sending out questions, write down the hazard you want to control. Note where it happens and what evidence you have, like incident and near-miss records. Vendors give better answers when they understand the specific problem. You can then score their responses against it.

Three practical rules make answers easier to compare:

  • Ask for documents. Test reports, certificates, sample contracts and data processing agreements resist inflation better than prose.
  • Fix the format. Set a word limit per answer and provide a pricing table so responses line up.
  • Agree on scoring first. Assign weights to each group before responses arrive, and have at least two people score them independently.

Whatever a vendor claims about outcomes at other sites, like reduced incidents, is still just a claim. Record it as such, and test what you can in a pilot.

Group 1: Fit and use cases (questions 1 to 5)

These questions check whether the product targets your actual hazard or just a nearby one.

# Question A good answer includes Red flags
1 Which of our listed hazards can your product detect today, and which detections are in beta or on the roadmap? A clear split between generally available, beta and planned, mapped to your hazards Roadmap items presented as available
2 Which customers use these detections in operations similar to ours (sector, site size, indoor or outdoor)? Named or anonymized examples with comparable conditions Only demo footage or unrelated sectors
3 What does your product not do well, and in which conditions do you advise against it? Specific limits, such as long distances, heavy occlusion or low light "No limitations"
4 How are site-specific rules, zones and exceptions configured, and who does it? A described process, with customer and vendor roles Every change needs a paid vendor request
5 What changes on our side (layout, procedures, staffing) have customers needed to make to get value? Honest examples of supervisor time and process changes "Plug and play" with no workload

Group 2: Detection performance and evidence (questions 6 to 11)

Precision measures the share of alerts that turn out to be real events. Recall tracks the share of real events the system actually catches [13]. Pushing one up usually pulls the other down. A single accuracy figure can look high even if a system misses rare events [13]. Ask for both metrics, broken down by detection type.

# Question A good answer includes Red flags
6 What are precision and recall for each proposed detection, and on what test data (hours, sites, lighting, camera angles)? Figures per detection with a described dataset and date One headline "accuracy" number
7 How does performance change at night, in rain, dust, glare, at long range or with partial occlusion? Results broken down by condition, or a plan to test them in your pilot No condition-level data
8 Have you tested whether detection rates differ by body size, skin tone, clothing, headwear, PPE color or mobility aids? What did you find? A described test and its results, including any gaps found "Our AI has no bias"
9 What alert volume per camera per shift should we expect, and how is it tuned down without losing important events? A typical range from comparable sites and a tuning method No figures
10 Will you support a pilot measured on our own footage, with our reviewers labeling alerts and staged tests for recall? Agreement, plus help with test design Results reported only from the vendor's dashboard
11 How often do models change, how is each update validated, and can we defer or roll back an update? Release notes, regression testing and customer control Silent updates with no notice

Question 8 matters because demographic differences in error rates are well documented in some computer vision tasks. A 2019 NIST study of 189 face recognition algorithms found demographic differentials in the majority of them, including higher false positive rates for some demographic groups [12]. Safety detections such as person or PPE detection are different tasks, but the study shows why vendors should test for such gaps instead of assuming they are absent.

Group 3: Deployment and architecture (questions 12 to 16)

# Question A good answer includes Red flags
12 Where does processing run (camera, edge device, on-premises server, cloud), and what data leaves the site, in what form? A data flow diagram showing video, clips, images and metadata Vague "secure cloud" answer
13 What are the minimum camera specifications, and which of our existing cameras qualify? A site survey method and clear specifications All cameras qualify without a survey
14 What bandwidth, power, rack space and network changes are required? Figures per camera and per site Requirements discovered after contract
15 How do you detect camera outages, blocked views, moved cameras or degraded image quality? Automatic health monitoring and alerts Customer must notice problems
16 What happens to detection and alerting if the internet connection or cloud service is down? Local buffering or edge alerting, and stated recovery behavior Total loss of alerts with no notice

Group 4: Data protection and worker privacy (questions 17 to 22)

In the UK and EU, a data protection impact assessment (DPIA) is required before processing that is likely to result in a high risk, which workplace video analytics will often be, and the ICO's guidance uses location tracking of employees as an example of processing that requires one [2]. Under the GDPR, biometric data used to uniquely identify a person is a special category with stricter conditions, and vendors acting as processors must sign terms meeting Article 28 [1].

# Question A good answer includes Red flags
17 What personal data does the system process at each stage, and does any feature use biometric identification such as face matching? A data inventory per feature, with identity features clearly marked Unclear whether faces are matched
18 Can faces and bodies be blurred or anonymized before data leaves the device, and can identity features be switched off entirely? Configurable anonymization at the edge and a way to verify it Anonymization only in the user interface
19 Do you use customer footage to train or improve models? Is it opt-in, how is it de-identified, and can we withdraw? Off by default, written opt-in, described de-identification, withdrawal process Training rights buried in terms of service
20 What are the default and configurable retention periods for video, clips, images and event data? Per data type, configurable by the customer One fixed retention period
21 Where is data hosted, which sub-processors are used, and will you sign our data processing agreement? Region options, a current sub-processor list, Article 28 terms [1] Refusal to share sub-processors
22 What documentation will you provide for our DPIA and worker consultation? A DPIA support pack and plain-language worker materials "That is the customer's job"

The ICO's monitoring guidance says employers must make sure workers are aware of what personal information monitoring collects and how, and must be clear about its purpose [3]. Vendor materials written for workers, as requested in question 22, help meet that expectation.

Group 5: AI governance and regulation (questions 23 to 27)

The EU AI Act already prohibits AI systems that infer the emotions of people at work, except for medical or safety reasons; that prohibition has applied since 2 February 2025 [4][6]. Where a system is high-risk under the Act, employers deploying it must inform workers' representatives and affected workers before putting it into service and assign competent human oversight [5]. Under the amended application dates in Article 113, obligations for high-risk systems listed in Annex III apply from 2 December 2027 [6]. Ask these questions now, because contracts signed today will run into that period.

# Question A good answer includes Red flags
23 Have you assessed whether any part of the product is a high-risk AI system under the EU AI Act? What is the conclusion and reasoning? A written assessment per feature "The AI Act does not apply to us" with no reasoning
24 Does any feature infer emotions, stress, fatigue or engagement from faces, voice or body language? A clear yes or no per feature, with the safety rationale for any exception claimed Fatigue or attention scoring with no legal analysis
25 What instructions for use, intended purpose statement and logging will you provide to support our deployer duties? Documents available now, plus log export Promised "when required"
26 Do you follow a recognized AI governance framework, such as the NIST AI Risk Management Framework or ISO/IEC 42001? Named framework and evidence of use, such as a certificate or internal policy [7][8] Framework named with no evidence
27 How do customers report errors or bias concerns, and how do you investigate and tell affected customers? A defined process with response times No process

Group 6: Security (questions 28 to 32)

# Question A good answer includes Red flags
28 Which security certifications or attestations do you hold, and what is their scope? Current ISO/IEC 27001 certificate or SOC 2 report, with scope covering the product and hosting [9][10] Certification of a data center only
29 How are edge devices hardened, patched and monitored, and how fast are critical vulnerabilities fixed? Stated patch timelines and secure boot or equivalent Manual patching on request
30 Who at your company can access our systems or footage remotely, and how is that access approved and logged? Named roles, customer approval, full logging Standing remote access for support staff
31 How is the system separated from our operational technology networks, and do you follow ISA/IEC 62443 guidance? Network segmentation design and familiarity with industrial security standards [11] Request for flat network access
32 How quickly will you notify us of a security incident affecting our data? A fixed number of hours in the contract "Promptly"

SOC 2 reports cover controls relevant to security, availability, processing integrity, confidentiality or privacy, and a given report need not cover all five [10]. Always check the report's scope and read any exceptions the auditor flagged.

Group 7: Workflow, integration and support (questions 33 to 36)

# Question A good answer includes Red flags
33 How are alerts routed, reviewed, dismissed and escalated, and can we cap alert volume per person? Configurable routing and caps, with review logged All alerts to everyone
34 Which video management and EHS systems do you integrate with, and can alerts create actions whose status flows back? Named integrations and a documented API Integration "on the roadmap"
35 Can we export all event data and clips in an open format at any time, at no extra cost? Self-service export in standard formats Export only at exit, for a fee
36 What onboarding, training and support do you provide, in which languages and time zones? Named service tiers and response times Support by email only

Group 8: Commercial terms, references and exit (questions 37 to 40)

# Question A good answer includes Red flags
37 What is the full three-year cost, including licenses, hardware, installation, support and annual price increases? A complete pricing table with caps on increases Hardware or support priced later
38 What are the pilot terms, and how do pilot fees convert into a production contract? Written pilot terms, acceptance criteria and conversion pricing Free pilot with no terms
39 Can you give three references with similar operations, one live for more than a year, and one customer that stopped using the product? All three, including the former customer Only recent or hand-picked references
40 What happens to our data and service if we leave, or if you are acquired or stop trading? Export, certified deletion, transition support and change-of-control terms No exit provisions

The European Commission's model contractual AI clauses, published in high-risk and non-high-risk versions for public buyers, offer a useful source of contract wording on transparency, human oversight and accuracy [14]. They leave out liability and payment terms. So, pair them with your standard commercial agreements.

How should answers be scored?

A simple scheme works well:

Score Meaning
0 No answer, refusal or irrelevant answer
1 Answer given with no supporting evidence
2 Clear answer with partial evidence
3 Clear answer backed by documents, test results or contract terms

Weight the groups to match your priorities. For a first deployment of video analytics, many buyers give the most weight to detection performance, data protection and three-year cost, and less to breadth of features. Treat some answers as pass or fail regardless of score, for example a refusal to switch off identity features or to put a breach notification period in the contract.

Shortlist two or three vendors and test them in a structured pilot with written acceptance criteria. The RFP tells you what a vendor says it can do. The pilot tells you whether it does so on your site.

Summary

A good RFP for AI safety technology asks for evidence in every area: fit with your hazards, precision and recall per detection in conditions like yours, deployment and resilience, data protection and worker privacy, AI governance and EU AI Act readiness, security, workflow and integration, and commercial and exit terms. The 40 questions here, each with what a good answer includes and the red flags to watch for, can be used in full for a formal tender or cut down to the most important for a smaller request. Agree weighted scoring before answers arrive, treat vendor outcome figures as claims, ask for a reference that left, and confirm the shortlist through a pilot measured on your own footage.

Frequently asked questions

+Do we need to ask all 40 questions?

No. Use the full list for a formal RFP covering several sites, and pick the ten to fifteen most relevant for a smaller request for information. The questions on detection evidence, data use for model training, emotion recognition, security attestations and exit terms are worth keeping in every version.

+How should we score vendor answers?

Agree weights for each group before issuing the RFP, then score each answer on a simple scale such as 0 to 3, where 0 means no answer or a refusal and 3 means a clear answer backed by documents. Score the evidence offered, and have at least two people score independently.

+What if a vendor says the information is confidential?

Offer a non-disclosure agreement for detailed test results, security reports and contracts. A vendor that will not share a SOC 2 report or test methodology under NDA is asking you to accept its claims on trust, and that should cost it points.

+Should the same questions apply to proximity warning and wearable vendors?

Most of them apply with small changes. Replace the camera questions with questions about tags, batteries, radio coverage and charging, and keep the questions on detection performance, data, security, workflow and exit.

Sources

  1. [1]Regulation (EU) 2016/679 (General Data Protection Regulation), EUR-Lex
  2. [2]ICO, When do we need to do a DPIA?
  3. [3]ICO, Monitoring workers: data protection and monitoring workers
  4. [4]AI Act, Article 5: Prohibited AI practices
  5. [5]AI Act, Article 26: Obligations of deployers of high-risk AI systems
  6. [6]AI Act, Article 113: Entry into force and application
  7. [7]NIST, AI Risk Management Framework
  8. [8]ISO/IEC 42001 Information technology, Artificial intelligence, Management system
  9. [9]ISO/IEC 27001 Information security management systems
  10. [10]AICPA, SOC 2 examinations
  11. [11]ISA, ISA/IEC 62443 series of standards
  12. [12]NIST, NIST study evaluates effects of race, age, sex on face recognition software (2019)
  13. [13]Google for Developers, Classification: accuracy, recall, precision and related metrics
  14. [14]European Commission Public Buyers Community, EU model contractual AI clauses

Get the free Safety Tech Buyer's Toolkit

A 15-page PDF: the public evidence checklist, a weighted vendor scorecard, 40 RFP questions and pilot acceptance criteria. Subscribe and download it straight away; then one email a month when we publish or update guidance. No vendor promotions. Unsubscribe any time.