Let’s start with the kind of question you are likely to ask yourself the first time you come across something new.

“What do I need a Benchmarking Engine for?”

A possible short answer is this: To efficiently and automatically identify opportunities for business performance improvement, customer/vendor satisfaction, and revenue generation.

Now for a more comprehensive answer…

A long time ago, in a galaxy far, far away…

 Long before Benchmarking Engines existed, people have relied on different types of computational engines, and it’s pretty much always the case that depending on what type of question is asked, a different type of engine is required.

Search Engines support the classical problems of Information Retrieval. Traditionally, they answer questions in the following general format:

“What/Where/When/Who is X?”

Similarly, Semantic Engines seek to understand the contextual meaning of words. They’re commonly associated with questions related to Natural Language Processing and Semantic/Sentiment analysis. For example:

“What does the word X mean?”
“What’s the sentiment associated with Z?”

Classification Engines help determine the best classification for something based on past trained observations.

“Which one is X? Is it a Y, Z, or W?”
“How likely is X to be a Y?”

Recommendation Engines provide answers better than engines designed for generic audiences. They use past behavior to provide recommendations based on what they predict that a user will like or not like, such as:

“What can I recommend to user X based on other similar users’ past behavior?”
“What can I recommend to user X based on other users that show similar past behavior?”
“What can I recommend to user X based on its own past behavior?”

Prediction Engines provide their best interpretation of what will happen next under certain future conditions. They’re commonly used for pricing/behavior prediction and to assist in business decision-making. For example:

“What will X likely be [if Y does happen] ?”
“What will X likely be [on Z date]?”
“What will X likely be [after N iterations]?”

Here’s some food for thought: If you were a business, what kind of engine would you use if you needed answers for the following questions?

“How are we doing?”
“Where could we improve?”
“What’s best in class?”

… and that’s what you need a Benchmarking Engine for.

Artificial Intelligence: It’s all about the facts

These days there’s a lot of talk about Artificial Intelligence, especially in everything related to Deep Learning, Neural Networks, and all they can do in terms of classification and pattern recognition. It’s a known fact that those techniques can be used to recognize people/cats/handwriting/text/speech/etc, but keep in mind that Artificial Intelligence is not all about creating classification models and neural networks. Classical AI systems are made up of a few different parts: knowledge bases, rules, and inference mechanisms. Those systems mimic the judgment of experts by leveraging known facts and by following well-defined sets of rules. Sometimes they even use fuzzy logic to adjust parameters on the fly.

Benchmarking Engines are a new type of computational engine designed and built with Artificial Intelligence principles. Benchmarking Engines work with facts, analyzing them in detail and pointing out insights that describe in perfect english what is special about each one of the benchmarked entities. This new type of engine uses combinatorial search algorithms to comb through every single combination of facts and identify the best insights that need to be highlighted to users.  For each insight, peer groups and interesting metrics are dynamically identified. Not imposed. So, no bias are introduced into the analysis.

The New Engine On The Block

Unlike traditional benchmarking techniques, Benchmarking Engines should NOT…

• … limit what data goes into a benchmarking project. The more, the merrier;
• … drive benchmarking projects into something expensive or lengthy;
• … pre-define what metrics or peer groups go into a benchmarking project;
• … deal with just numbers. Some facts are larger than numbers;
• … be model-based because it takes time to learn about new subjects;
• … benefit just a single benchmarked entity. A scalable approach is desired;
• … assume everyone should access everything. Data anonymization is a must;

But how can a business really ($$$) benefit from a Benchmarking Engine?

Here’s one use case: Let’s say you’re a B2B SaaS company and you regularly collect a lot of performance/operational data about how your customers use your product. What if you could provide them with reports describing how they’re doing and where they can improve based on a comparison analysis across all of your customers?

Here’s another scenario: Let’s assume you’re a company that has hundreds of vendors. What if you could provide each one of them with a tool that describes how they’re doing and where they can improve when compared to all of your other vendors?

Best of all, what if you could charge all of them for such insights? and therefore create a new revenue stream for your company using data you already have?

Learn more about Benchmarking Engines.

Cheers,
~ Andre Lessa