An Italian court determined that companies can be held liable even if an algorithm unintentionally discriminates against a protected group.
An algorithm used by the popular European food delivery app Deliveroo to rank and offer shifts to riders is discriminatory, an Italian court ruled late last week, in what some experts are calling a historic decision for the gig economy. The case was brought by a group of Deliveroo riders backed by CGIL, Italy’s largest trade union.
A markedly detailed ordinance written by presiding judge Chiara Zompi gives an intimate look at one of many often secretive algorithms used by gig platforms to micromanage workers and which can have profound impacts on their livelihoods.
While machine-learning algorithms are central to Deliveroo’s entire business model, the particular algorithm examined by the court allegedly was used to determine the “reliability” of a rider. According to the ordinance, if a rider failed to cancel a shift pre-booked through the app at least 24 hours before its start, their “reliability index” would be negatively affected. Since riders deemed more reliable by the algorithm were first to be offered shifts in busier timeblocks, this effectively meant that riders who can’t make their shifts—even if it’s because of a serious emergency or illness—would have fewer job opportunities in the future.