Artificial intelligence (AI) is rapidly becoming an indispensable part of many businesses. However, it is important to ask some tough questions before hiring an AI company. For example, if you are not aware of the underlying AI models that are being used in a product, you may not be able to anticipate its limitations. Additionally, if you are not sure how your data will be handled, you may be putting your company at risk. This blog provides some advice on how to test AI systems to make sure that they meet your needs.

1. Which concrete AI models do you use in your product? Who is their owner, and what are their license terms?

Providers of AI-products can’t keep these things in secret, since each underlying model has its own specific constraints and terms of use. And you, as a customer, must perform a background check to avoid any surprises.

For example, LLama2 (and derivatives) could be used only with English language, while ChatGPT can have degraded availability during the peak hours.

2. Can you guarantee that 3rd parties will not see my data?

For example, if GPT/LLM models are provided by Microsoft Azure, then all requests are stored there for 30 days. An exception can be made for the customers that have gone through the special approval process.

Of course, it is better when all models run on the premises under your control completely.

3. You are offering data anonymisation for GDPR compliance. Can you guarantee (in writing) that your process is 100% efficient? Better yet, can you demo the process to us on our data?

It is fairly easy to promise data anonymisation. However, it is nearly impossible to ensure 100% removal of Personally Identifiable Information (PII), especially from the unstructured data. Automated processes can miss some nuances that are specific to your business.

Best approach here is to talk through the nuances and evaluate data anonymisation of your own data.

4. Do you offer AI-driven information retrieval?

Let’s test your system on our data together! Our employees will gather a a bunch of documents for 1000 pages and will prepare a list of 20 questions. 10 of these questions will have answers on specific pages, while 10 questions will have no answers.

This document package could be prepared by your own employees, the ones that will be using the new system on a day-to-day basis. Nobody would also be better for coming up with a list of domain-specific questions.

To keep the test fair, make sure that the provider of AI-driven product does not see the documents and questions before the test.

5. For an AI system, it is fine to make mistakes. Even OpenAI ChatGPT makes them and learns from them. But how can your system capture user feedback and learn from it? Demo this to us!

You will have to work with the new AI product for many years. This cooperation will be more valuable for you, if the system could not only adapt to your business, but also adapt to any future changes.

These 5 questions will be very inconvenient for the providers of AI-driven products that try to sell thin wrappers around ChatGPT or LLamaIndex as a “GDPR-compliant technological breakthrough”. Vendors of AI products that have invested a lot of thought, expertise and effort into their products - would be delighted to talk about these topics.

This also includes a Trustbit. If you are interviewing AI products for joining your business, consider inviting us over, too!