When working with System One models like Jev, we may need decision-making algorithms such as Multi-Criteria Decision Making (MCDA).
Unlike ML, MCDA uses predefined criteria and weights. This usually makes its decisions more transparent and easier to understand, and it can work even with a small amount of data. Common MCDA methods include WSM and TOPSIS. However, MCDA depends on subjective weights and criteria, adapts poorly to changes, and becomes more complex when there are many alternatives. ML, on the other hand, learns from data, finds patterns, works well with large and complex datasets, and can adapt when new data arrives. However, it depends on data quality, requires more computing resources, and is often harder to interpret.
WSM - Weighted Sum Method
WSM is a method where each criterion gets a weight. The value of an alternative for each criterion is multiplied by its weight, and all results are added together. The higher the final score, the better the alternative.
https://www.geeksforgeeks.org/dsa/weighted-sum-method-multi-criteria-decision-making/
import numpy as np
# Alternatives matrix.
# Each row represents one student.
# Column 0: CGPA, or average grade.
# Column 1: Stipend, or required cost.
# Column 2: Technical Exam score.
# Column 3: Aptitude Test score.
X = np.array([
[9.0, 12000, 72, 3],
[7.6, 8500, 68, 3],
[8.2, 9500, 63, 2],
[8.5, 10000, 70, 4],
[9.3, 14000, 72, 4]
])
# Criterion weights: CGPA = 30%, Stipend = 20%, Technical Exam = 25%, Aptitude Test = 25%.
weights = np.array([0.3, 0.2, 0.25, 0.25])
# True means a higher value is better, and False means a lower value is better.
benefit = [True, False, True, True]
# Copy the original data for normalization.
N = X.copy()
for j in range(X.shape[1]):
if benefit[j]:
# For a criterion that must be maximized, divide each value by the maximum value in the column.
# Example result: [0.968, 0.817, 0.882, 0.914, 1.000].
N[:, j] = X[:, j] / X[:, j].max()
else:
# For a criterion that must be minimized, divide the minimum value by each value in the column.
# Stipend example: [12000, 8500, 9500, 10000, 14000].
# 8500 / 12000 = 0.708.
# 8500 / 8500 = 1.000.
# 8500 / 9500 = 0.895.
# 8500 / 10000 = 0.850.
# 8500 / 14000 = 0.607.
# Example result: [0.708, 1.000, 0.895, 0.850, 0.607].
# The smallest value gets 1 because a lower value is better for this criterion.
N[:, j] = X[:, j].min() / X[:, j]
# Multiply each normalized value by its criterion weight and add all values together.
scores = N @ weights
print("Scores:", scores)
# Find the index of the highest final score and add 1 because Python indexes start from 0.
best = scores.argmax() + 1
print("Best student:", best)
And here is an additional example of using WSM specifically with Jev - incident response. Suppose that after a new release, checkout errors increase, and we need to choose between rolling back the release, disabling the faulty feature via a feature flag, or preparing a hotfix. Jev can evaluate each option based on criteria such as recovery speed, safety, customer impact, and implementation ease, while WSM combines those scores using predefined weights. Hard constraints can also be applied before selecting an action - for example, excluding options whose safety score falls below an acceptable threshold.
import os
import requests
API_KEY = os.environ["OPENROUTER_API_KEY"]
state = """
After release v2.4, checkout errors increased to 18%.
The previous version is stable.
The faulty feature can be disabled with a feature flag.
A hotfix is estimated to require code changes and a new deployment.
"""
actions = [
"rollback",
"disable_feature",
"hotfix",
]
criteria = {
"recovery_speed": {
"instructions": "How quickly would {action} restore normal service?",
"criteria": ["very_slow", "slow", "medium", "fast", "very_fast"],
},
"safety": {
"instructions": "How safe is {action} in this incident?",
"criteria": ["very_risky", "risky", "neutral", "safe", "very_safe"],
},
"customer_impact": {
"instructions": "How effectively would {action} reduce customer impact?",
"criteria": ["very_low", "low", "medium", "high", "very_high"],
},
"implementation_ease": {
"instructions": "How easy is {action} to execute?",
"criteria": ["very_hard", "hard", "medium", "easy", "very_easy"],
},
}
weights = {
"recovery_speed": 0.40, # Recovery is the most important factor
"safety": 0.30,
"customer_impact": 0.20,
"implementation_ease": 0.10,
}
results = {}
for action in actions:
questions = {}
for criterion, config in criteria.items():
question_name = f"{action}_{criterion}"
questions[question_name] = {
"type": "score",
"instructions": config["instructions"].format(action=action),
"criteria": config["criteria"],
}
response = requests.post(
"https://openrouter.ai/api/alpha/decisions",
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
},
json={
"model": "~typesafe/jev-latest",
"state": state,
"questions": questions,
},
)
answers = response.json()["answers"]
scores = {
criterion: answers[f"{action}_{criterion}"]["score"] for criterion in criteria
}
wsm_score = sum(scores[criterion] * weights[criterion] for criterion in weights)
normalized_wsm = wsm_score / 4 # Normalize to a 0-1 scale (assuming max score is 4)
results[action] = {
"scores": scores,
"wsm": wsm_score,
"utility": normalized_wsm,
}
MIN_SAFETY_SCORE = 2
eligible_results = {
action: result
for action, result in results.items()
if result["scores"]["safety"] >= MIN_SAFETY_SCORE
}
if not eligible_results:
best_action = "human_review"
else:
best_action = max(
eligible_results,
key=lambda action: results[action]["wsm"],
)
print(best_action)
In my case, disable_feature was chosen.
TOPSIS - Technique for Order Preference by Similarity to Ideal Solution
In TOPSIS, the values for all criteria are first normalized and then multiplied by their weights. After that, the ideal best and ideal worst values are found for each criterion. The distance from each alternative to the ideal best and ideal worst solutions is calculated, and then the closeness score is found. The higher this score, the better the alternative.
import numpy as np
import pandas as pd
# Input data.
df = pd.DataFrame({
"Phone": ["A", "B", "C", "D"],
"RAM": [4, 6, 6, 8],
"Memory": [128, 64, 128, 256],
"Display": [6.5, 6.4, 6.8, 7.0],
"Battery": [3500, 3800, 4200, 5000],
"Price": [15000, 16000, 19000, 25000]
})
matrix = df[
["RAM", "Memory", "Display", "Battery", "Price"]
].to_numpy(dtype=float)
# Criterion weights: RAM = 10%, Memory = 10%, Display = 20%, Battery = 20%, Price = 40%.
# A larger weight means the criterion has more influence on the final result.
# Price = 0.4 means that price has more influence than RAM = 0.1.
weights = np.array([0.1, 0.1, 0.2, 0.2, 0.4])
# +1 means a higher value is better, and -1 means a lower value is better.
impacts = np.array([1, 1, 1, 1, -1])
# Normalization is needed because the criteria use different scales.
# Example scales: RAM is about 4-8, Battery is about 3500-5000, and Price is about 15000-25000.
column_norms = np.sqrt(np.sum(matrix ** 2, axis=0))
normalized = matrix / column_norms
# Example normalized matrix.
# normalized = [
# [0.3244, 0.4000, 0.4866, 0.4203, 0.3916], # A
# [0.4867, 0.2000, 0.4791, 0.4564, 0.4177], # B
# [0.4867, 0.4000, 0.5090, 0.5044, 0.4961], # C
# [0.6489, 0.8000, 0.5240, 0.6005, 0.6527] # D
# ]
# Apply the weights so more important criteria have more influence on TOPSIS distances.
weighted = normalized * weights
# Example weighted matrix.
# weighted = [
# [0.03, 0.04, 0.09, 0.08, 0.15], # A
# [0.05, 0.02, 0.08, 0.09, 0.16], # B
# [0.05, 0.04, 0.09, 0.10, 0.19], # C
# [0.07, 0.08, 0.10, 0.12, 0.25], # D
# ]
# Ideal Best uses MAX for positive criteria and MIN for negative criteria such as Price.
best = np.where(
impacts == 1,
weighted.max(axis=0),
weighted.min(axis=0)
)
# Example Ideal Best: [0.07, 0.08, 0.10, 0.12, 0.15].
# Ideal Worst uses the opposite values.
worst = np.where(
impacts == 1,
weighted.min(axis=0),
weighted.max(axis=0)
)
# Example Ideal Worst: [0.03, 0.02, 0.08, 0.08, 0.25].
# A good alternative should be close to Ideal Best and far from Ideal Worst.
distance_to_best = np.linalg.norm(weighted - best, axis=1)
distance_to_worst = np.linalg.norm(weighted - worst, axis=1)
# TOPSIS score = distance_to_worst / (distance_to_best + distance_to_worst).
# A score closer to 1 is better.
score = (
distance_to_worst /
(distance_to_best + distance_to_worst)
)
df["Distance Best"] = distance_to_best
df["Distance Worst"] = distance_to_worst
df["TOPSIS Score"] = score
print("\nDistances and TOPSIS Score:")
print(
df[
["Phone", "Distance Best", "Distance Worst", "TOPSIS Score"]
].round(4)
)
# Example result:
# Phone Distance Best Distance Worst TOPSIS Score
# 0 A 0.0633 0.1063 0.6269
# 1 B 0.0699 0.0957 0.5778
# 2 C 0.0631 0.0701 0.5260
# 3 D 0.1044 0.0777 0.4265
# The alternative with the highest TOPSIS Score gets first place.
# A = 0.6269.
Choosing a Method
| Method | When to use | Example: supplier selection |
|---|---|---|
| WSM | Use it when the weights are already known, the criteria can be normalized, and simple compensation is acceptable, meaning a poor result for one criterion can be balanced by a good result for another. | Score = Price×0.2 + Quality×0.4 + Delivery×0.25 + Support×0.15 |
| TOPSIS | Use it when the weights are already known and you want an alternative that is close to the best values for all criteria and far from the worst values. | Choose the supplier whose Price/Quality/Delivery/Support profile is closest to the ideal profile. |
