Connect with us

Technology

Perceptron: AI bias can arise from annotation instructions

Published

on

Research in the field of machine learning and AI, now a key technology in practically every industry and company, is far too voluminous for anyone to read it all. This column, Perceptron (previously Deep Science), aims to collect some of the most relevant recent discoveries and papers — particularly in, but not limited to, artificial intelligence — and explain why they matter.

This week in AI, a new study reveals how bias, a common problem in AI systems, can start with the instructions given to the people recruited to annotate data from which AI systems learn to make predictions. The coauthors find that annotators pick up on patterns in the instructions, which condition them to contribute annotations that then become over-represented in the data, biasing the AI system toward these annotations.

Many AI systems today “learn” to make sense of images, videos, text, and audio from examples that have been labeled by annotators. The labels enable the systems to extrapolate the relationships between the examples (e.g., the link between the caption “kitchen sink” and a photo of a kitchen sink) to data the systems haven’t seen before (e.g., photos of kitchen sinks that weren’t included in the data used to “teach” the model).

This works remarkably well. But annotation is an imperfect approach — annotators bring biases to the table that can bleed into the trained system. For example, studies have shown that the average annotator is more likely to label phrases in African-American Vernacular English (AAVE), the informal grammar used by some Black Americans, as toxic, leading AI toxicity detectors trained on the labels to see AAVE as disproportionately toxic.

As it turns out, annotators’ predispositions might not be solely to blame for the presence of bias in training labels. In a preprint study out of Arizona State University and the Allen Institute for AI, researchers investigated whether a source of bias might lie in the instructions written by data set creators to serve as guides for annotators. Such instructions typically include a short description of the task (e.g. “Label all birds in these photos”) along with several examples.

Parmar et al.

Image Credits: Parmar et al.

The researchers looked at 14 different “benchmark” data sets used to measure the performance of natural language processing systems, or AI systems that can classify, summarize, translate, and otherwise analyze or manipulate text. In studying the task instructions provided to annotators that worked on the data sets, they found evidence that the instructions influenced the annotators to follow specific patterns, which then propagated to the data sets. For example, over half of the annotations in Quoref, a data set designed to test the ability of AI systems to understand when two or more expressions refer to the same person (or thing), start with the phrase “What is the name,” a phrase present in a third of the instructions for the data set.

The phenomenon, which the researchers call “instruction bias,” is particularly troubling because it suggests that systems trained on biased instruction/annotation data might not perform as well as initially thought. Indeed, the coauthors found that instruction bias overestimates the performance of systems and that these systems often fail to generalize beyond instruction patterns.

The silver lining is that large systems, like OpenAI’s GPT-3, were found to be generally less sensitive to instruction bias. But the research serves as a reminder that AI systems, like people, are susceptible to developing biases from sources that aren’t always obvious. The intractable challenge is discovering these sources and mitigating the downstream impact.

In a less sobering paper, scientists hailing from Switzerland concluded that facial recognition systems aren’t easily fooled by realistic AI-edited faces. “Morphing attacks,” as they’re called, involve the use of AI to modify the photo on an ID, passport, or other form of identity document for the purposes of bypassing security systems. The coauthors created “morphs” using AI (Nvidia’s StyleGAN 2) and tested them against four state-of-the art facial recognition systems. The morphs didn’t post a significant threat, they claimed, despite their true-to-life appearance.

Elsewhere in the computer vision domain, researchers at Meta developed an AI “assistant” that can remember the characteristics of a room, including the location and context of objects, to answer questions. Detailed in a preprint paper, the work is likely a part of Meta’s Project Nazare initiative to develop augmented reality glasses that leverage AI to analyze their surroundings.

Meta egocentric AI

Image Credits: Meta

The researchers’ system, which is designed to be used on any body-worn device equipped with a camera, analyzes footage to construct “semantically rich and efficient scene memories” that “encode spatio-temporal information about objects.” The system remembers where objects are and when the appeared in the video footage, and moreover grounds answers to questions a user might ask about the objects into its memory. For example, when asked “Where did you last see my keys?,” the system can indicate that the keys were on a side table in the living room that morning.

Meta, which reportedly plans to release fully-featured AR glasses in 2024, telegraphed its plans for “egocentric” AI last October with the launch of Ego4D, a long-term “egocentric perception” AI research project. The company said at the time that the goal was to teach AI systems to — among other tasks — understand social cues, how an AR device wearer’s actions might affect their surroundings, and how hands interact with objects.

From language and augmented reality to physical phenomena: an AI model has been useful in an MIT study of waves — how they break and when. While it seems a little arcane, the truth is wave models are needed both for building structures in and near the water, and for modeling how the ocean interacts with the atmosphere in climate models.

Image Credits: MIT

Normally waves are roughly simulated by a set of equations, but the researchers trained a machine learning model on hundreds of wave instances in a 40-foot tank of water filled with sensors. By observing the waves and making predictions based on empirical evidence, then comparing that to the theoretical models, the AI aided in showing where the models fell short.

A startup is being born out of research at EPFL, where Thibault Asselborn’s PhD thesis on handwriting analysis has turned into a full-blown educational app. Using algorithms he designed, the app (called School Rebound) can identify habits and corrective measures with just 30 seconds of a kid writing on an iPad with a stylus. These are presented to the kid in the form of games that help them write more clearly by reinforcing good habits.

“Our scientific model and rigor are important, and are what set us apart from other existing applications,” said Asselborn in a news release. “We’ve gotten letters from teachers who’ve seen their students improve leaps and bounds. Some students even come before class to practice.”

Image Credits: Duke University

Another new finding in elementary schools has to do with identifying hearing problems during routine screenings. These screenings, which some readers may remember, often use a device called a tympanometer, which must be operated by trained audiologists. If one is not available, say in an isolated school district, kids with hearing problems may never get the help they need in time.

Samantha Robler and Susan Emmett at Duke decided to build a tympanometer that essentially operates itself, sending data to a smartphone app where it is interpreted by an AI model. Anything worrying will be flagged and the child can receive further screening. It’s not a replacement for an expert, but it’s a lot better than nothing and may help identify hearing problems much earlier in places without the proper resources.

Technology

Amazon-owned MGM makes a viral video show with surveillance footage from Amazon-owned Ring

Published

on

MGM (which is owned by Amazon) is making a viral video show based on footage from Ring security cameras (also owned by Amazon). The syndicated television show, “Ring Nation,” is poised to be a modern-day, surveillance-tinged spin on “America’s Funniest Home Videos” with Wanda Sykes as host.

According to a report in Deadline, the show will feature Ring footage of “neighbors saving neighbors, marriage proposals, military reunions and silly animals.” Ring is also known for activities like accidentally leaking people’s home addresses and handing over footage to the government without users’ permission.

Between January and July of this year, Amazon shared ring doorbell footage with U.S. authorities 11 times without the device owner’s consent. Ring has been critiqued for working unusually closely with at least 2,200 police departments around the United States, allowing police to request video doorbell camera footage from homeowners through Ring’s Neighbors app. Like Citizen and Nextdoor, the Neighbors app tracks local crime and allows users to comment anonymously — plus, Ring’s police partners can publicly request video footage on the app.

An Amazon-owned police surveillance network is bad enough, but Neighbors users have also faced repeated safety and security issues.

An executive at MGM, Barry Poznick, praised the new show: “From the incredible, to the hilarious and uplifting must-see viral moments from around the country every day, Ring Nation offers something for everyone watching at home.”

But perhaps what viewers at home really want is data privacy.

Ring only started disclosing its connections with law enforcement after fielding demands for transparency from the U.S. government. In a 2019 letter, Senator Ed Markey (D-MA) said that the company’s relationship with police forces raise civil liberties concerns.

“The integration of Ring’s network of cameras with law enforcement offices could easily create a surveillance network that places dangerous burdens on people of color and feeds racial anxieties in local communities,” Sen. Markey wrote. “In light of evidence that existing facial recognition technology disproportionately misidentifies African Americans and Latinos, a product like this has the potential to catalyze racial profiling and harm people of color.”

Amazon bought the smart video doorbell company in 2018 for $1 billion, then bought MGM for $8.5 billion earlier this year. Now, these two investments — which seemingly have nothing to do with each other — are merging to create a late-capitalist dystopian spectacular that we couldn’t have imagined in our worst nightmares. Amazon also just spent $1.7 billion on iRobot, maker of the Roomba vacuum, but we will not dare to imagine how that acquisition may one day inspire a horrifying TV show.

Continue Reading

Technology

Aramco’s Prosperity7 powers AI drug firm Insilico’s $95M round

Published

on

Hong Kong-based drug discovery and development company Insilico has secured fresh capital at a time that its CEO described as a “biotech winter.”

The firm has raised $35 million on the heels of its last tranche in June, bringing its total Series D investment to $95 million. The new round was “oversubscribed”, the firm’s founder and CEO Alex Zhavoronkov told TechCrunch, declining to disclose the company’s valuation.

Prosperity7, the venture capital arm of Saudi Arabia’s state oil company Aramco, led the new capital infusion. The fund has been actively scouring for opportunities in and around China that can scale globally and particularly in the Middle East.

Insilico, which operates R&D teams across Hong Kong, Shanghai, and New York, seems to be a good fit for Prosperity7.

“Prosperity7 inspired us to look into sustainable chemistry,” said Zhavoronkov. Insilico uses machine learning to identify potential drug targets and eventually create the drug. The same technology can also be applied to find novel and useful molecules for sustainable chemistry, an emerging area to which Aramco has devoted much effort, the founder explained.

Sustainable chemistry, as defined by OECD, is “a scientific concept that seeks to improve the efficiency with which natural resources are used to meet human needs for chemical products and services.” It “encompasses the design, manufacture, and use of efficient, effective, safe and more environmentally benign chemical products and processes.”

Other investors from the round include an unnamed “large, diversified asset management firm on the U.S. West Coast,” and an assortment of financial and strategic investors like BHR Partners, Warburg Pincus, B Capital Group, Qiming Venture Partners, Deerfield, Wilson Sonsini Goodrich & Rosati, BOLD Capital Partners, and Pavilion Capital.

Zhavoronkov himself also invested in the Series D financing.

When asked why the company straddles China and the U.S., the founder compared the drug discovery space to the early semiconductor industry where research was done mostly in the U.S. while hardware production happened in China.

AI drug discovery relies on a massive amount of investment in so-called contract research organizations (CROs), which provide support to pharmaceutical or medical device companies in the form of outsourcing. China, exemplified by cities like Wuxi, has in recent years emerged as a popular CRO hub for international pharma companies.

The founder was also keen to speak about the company’s new dual-CEO structure. He recently promoted GSK veteran Dr. Feng Ren to be his co-CEO, who is now overseeing Insilico’s R&D and drug business, while Zhavoronkov focuses on the firm’s AI platform.

“Ren generates a lot of proprietary data for us to train AI to do better than humans. We can use this internally for drug discovery and then export this tech to the rest of the industry,” Zhavoronkov said.

Continue Reading

Technology

Egyptian startup Convertedin raises $3M, caters to e-commerce brands in MENA and Latin America

Published

on

Convertedin, an Egyptian startup that operates a marketing operating system for e-commerce brands, has raised $3 million in a seed round led by Saudi Arabia-headquartered Merak Capital.

Other participating investors include 500 Global and MSAS. The company, in a statement, said it plans to utilize the funds for strategic hiring and further development of its platform.

When brands shift to e-commerce sales, they operate with vast amounts of fragmented data that need to be unified to drive informed decisions and growth. As such, platforms like Convertedin become essential because it caters to brands and businesses with one, some, or all of these objectives: drive personalized and scalable campaigns, convert customers, achieve measurable results and grow revenue.

CEO Mohamed Fergany founded the company with Mohamed Atef and Mustafa Raslan in 2019 after working with several brands in companies such as Speakol Ads and Vodafone. His time as an employee opened his eyes to the opportunity of helping offline stores retarget and retain their customers online while finding new ones to shop at their stores offline.

“If you work into IKEA and they take your phone number down. After that, our engine works to find a similar product you might buy and we retarget you online. If you went back to IKEA for that product, we can calculate the cost of online conversion,” the chief executive said in the interview. “This was the main idea at this time as we saw a huge problem where there was no analytics platform for the offline store or a retargeting mechanism.”

As the pandemic hit and offline stores were forced to close their doors, many of these brands turned to e-commerce, and as a result, Convertedin took its business online too.

Fergany argues that though online brands use CRM software to gather data, they do not utilize most of it. So Convertedin offers a solution where they can use their data best. It plugs into more than 10 major e-commerce platforms and ad networks — and brands, once connected, can place customers into different segments such as high- and low-value and categories like those looking for specific products and use these insights to create personalized multi-channel marketing and drive various campaigns on social media, SMS, email, search and other channels while having the ability to track and attribute revenue conversion.

Convertedin says SMB e-commerce marketers that use its platform increase their return on ad spend (ROAS) by 2x and reduce customer acquisition costs (CAC) by 40%. So far, the company partners with media buying and advertising agencies and works with over 100 local and multinational brands across Africa, the Middle East and South America in the automotive, healthcare and technology industries. Convertedin’s revenues from these businesses have been growing in “double-digits” month-over-month, Fergany said.

The three-year-old Egypt-headquartered company also has offices in Saudi Arabia and Brazil; it just recently opened one in the latter. The South American market is enormous, with e-commerce revenues reaching $160 billion by 2025 from over 200 million users. As a result, Convertedin plans to make its services available in Portuguese — in addition to English and Arabic — for brands in Brazil and also Mexico, another South American market. Fergany also said Convertedin is eyeing South Africa and India too.

“We focus on emerging markets and if you look at it from healthy unit economics, we can sell easily in those countries because there is low competition there,” said the CEO on the expansion to five new markets, including Saudi Arabia. “And customer acquisition cost is low compared to the U.S. or Europe markets.” The new investment will help Convertedin with this expansion in addition to R&D and hiring.

In a statement, Ahmed Aljibreen, partner at lead investor Merak Capital, addressing his firm’s investment, said the ever-changing landscape of digital marketing platforms adds a new layer of challenges for e-commerce companies — and that Convertedin solves that. Hence, the reason why Merak Capital backed the firm. “We are excited to back Convertedin, a martech company that has built a state-of-the-art platform to simplify digital marketing, improve customer acquisition and drive growth for its clients. Convertedin is led by a world-class team in which we have tremendous confidence as the company embarks on its next stage of growth in MENA and Latin America.”

Continue Reading

Trending