OpenAI’s attempts to watermark AI text hit limits
Did a human write that, or ChatGPT? It can be hard to tell — perhaps too hard, its creator OpenAI thinks, which is why it is working on a way to “watermark” AI-generated content.
In a lecture at the University of Austin, computer science professor Scott Aaronson, currently a guest researcher at OpenAI, revealed that OpenAI is developing a tool for “statistically watermarking the outputs of a text [AI system].” Whenever a system — say, ChatGPT — generates text, the tool would embed an “unnoticeable secret signal” indicating where the text came from.
OpenAI engineer Hendrik Kirchner built a working prototype, Aaronson says, and the hope is to build it into future OpenAI-developed systems.
“We want it to be much harder to take [an AI system’s] output and pass it off as if it came from a human,” Aaronson said in his remarks. “This could be helpful for preventing academic plagiarism, obviously, but also, for example, mass generation of propaganda — you know, spamming every blog with seemingly on-topic comments supporting Russia’s invasion of Ukraine without even a building full of trolls in Moscow. Or impersonating someone’s writing style in order to incriminate them.”
Why the need for a watermark? ChatGPT is a strong example. The chatbot developed by OpenAI has taken the internet by storm, showing an aptitude not only for answering challenging questions but writing poetry, solving programming puzzles and waxing poetic on any number of philosophical topics.
While ChatGPT is highly amusing — and genuinely useful — the system raises obvious ethical concerns. Like many of the text-generating systems before it, ChatGPT could be used to write high-quality phishing emails and harmful malware, or cheat at school assignments. And as a question-answering tool, it’s factually inconsistent — a shortcoming that led programming Q&A site Stack Overflow to ban answers originating from ChatGPT until further notice.
To grasp the technical underpinnings of OpenAI’s watermarking tool, it’s helpful to know why systems like ChatGPT work as well as they do. These systems understand input and output text as strings of “tokens,” which can be words but also punctuation marks and parts of words. At their cores, the systems are constantly generating a mathematical function called a probability distribution to decide the next token (e.g., word) to output, taking into account all previously-outputted tokens.
In the case of OpenAI-hosted systems like ChatGPT, after the distribution is generated, OpenAI’s server does the job of sampling tokens according to the distribution. There’s some randomness in this selection; that’s why the same text prompt can yield a different response.
OpenAI’s watermarking tool acts like a “wrapper” over existing text-generating systems, Aaronson said during the lecture, leveraging a cryptographic function running at the server level to “pseudorandomly” select the next token. In theory, text generated by the system would still look random to you or I, but anyone possessing the “key” to the cryptographic function would be able to uncover a watermark.
“Empirically, a few hundred tokens seem to be enough to get a reasonable signal that yes, this text came from [an AI system]. In principle, you could even take a long text and isolate which parts probably came from [the system] and which parts probably didn’t.” Aaronson said. “[The tool] can do the watermarking using a secret key and it can check for the watermark using the same key.”
Watermarking AI-generated text isn’t a new idea. Previous attempts, most rules-based, have relied on techniques like synonym substitutions and syntax-specific word changes. But outside of theoretical research published by the German institute CISPA last March, OpenAI’s appears to be one of the first cryptography-based approaches to the problem.
When contacted for comment, Aaronson declined to reveal more about the watermarking prototype, save that he expects to co-author a research paper in the coming months. OpenAI also declined, saying only that watermarking is among several “provenance techniques” it’s exploring to detect outputs generated by AI.
Unaffiliated academics and industry experts, however, shared mixed opinions. They note that the tool is server-side, meaning it wouldn’t necessarily work with all text-generating systems. And they argue that it’d be trivial for adversaries to work around.
“I think it would be fairly easy to get around it by rewording, using synonyms, etc.,” Srini Devadas, a computer science professor at MIT, told TechCrunch via email. “This is a bit of a tug of war.”
Jack Hessel, a research scientist at the Allen Institute for AI, pointed out that it’d be difficult to imperceptibly fingerprint AI-generated text because each token is a discrete choice. Too obvious a fingerprint might result in odd words being chosen that degrade fluency, while too subtle would leave room for doubt when the fingerprint is sought out.
Yoav Shoham, the co-founder and co-CEO of AI21 Labs, an OpenAI rival, doesn’t think that statistical watermarking will be enough to help identify the source of AI-generated text. He calls for a “more comprehensive” approach that includes differential watermarking, in which different parts of text are watermarked differently, and AI systems that more accurately cite the sources of factual text.
This specific watermarking technique also requires placing a lot of trust — and power — in OpenAI, experts noted.
“An ideal fingerprinting would not be discernable by a human reader and enable highly confident detection,” Hessel said via email. “Depending on how it’s set up, it could be that OpenAI themselves might be the only party able to confidently provide that detection because of how the ‘signing’ process works.”
In his lecture, Aaronson acknowledged the scheme would only really work in a world where companies like OpenAI are ahead in scaling up state-of-the-art systems — and they all agree to be responsible players. Even if OpenAI were to share the watermarking tool with other text-generating system providers, like Cohere and AI21Labs, this wouldn’t prevent others from choosing not to use it.
“If [it] becomes a free-for-all, then a lot of the safety measures do become harder, and might even be impossible, at least without government regulation,” Aaronson said. “In a world where anyone could build their own text model that was just as good as [ChatGPT, for example] … what would you do there?”
That’s how it’s played out in the text-to-image domain. Unlike OpenAI, whose DALL-E 2 image-generating system is only available through an API, Stability AI open-sourced its text-to-image tech (called Stable Diffusion). While DALL-E 2 has a number of filters at the API level to prevent problematic images from being generated (plus watermarks on images it generates), the open source Stable Diffusion does not. Bad actors have used it to create deepfaked porn, among other toxicity.
For his part, Aaronson is optimistic. In the lecture, he expressed the belief that, if OpenAI can demonstrate that watermarking works and doesn’t impact the quality of the generated text, it has the potential to become an industry standard.
Not everyone agrees. As Devadas points out, the tool needs a key, meaning it can’t be completely open source — potentially limiting its adoption to organizations that agree to partner with OpenAI. (If the key were to be made public, anyone could deduce the pattern behind the watermarks, defeating their purpose.)
But it might not be so far-fetched. A representative for Quora said the company would be interested in using such a system, and it likely wouldn’t be the only one.
“You could worry that all this stuff about trying to be safe and responsible when scaling AI … as soon as it seriously hurts the bottom lines of Google and Meta and Alibaba and the other major players, a lot of it will go out the window,” Aaronson said. “On the other hand, we’ve seen over the past 30 years that the big Internet companies can agree on certain minimal standards, whether because of fear of getting sued, desire to be seen as a responsible player, or whatever else.”
The market has changed, but super-voting shares are here to stay, says Mr. IPO
Yesterday, the ride-sharing company Lyft said its two co-founders, John Zimmer and Logan Green, are stepping down from managing the company’s day-to-day operations, though they are retaining their board seats. According to a related regulatory filing, they actually need to hang around as “service providers” to receive their original equity award agreements. (If Lyft is sold or they’re fired from the board, they’ll see “100% acceleration” of these “time-based” vesting conditions.)
As with so many founders who’ve used multi-class voting structures in recent years to cement their control, their original awards were fairly generous. When Lyft went public in 2019, its dual-class share structure provided Green and Zimmer with super-voting shares that entitled them to 20 votes per share in perpetuity, meaning not just for life but also for a period of nine to 18 months after the passing of the last living co-founder, during which time a trustee would retain control.
It all seemed a little extreme, even as such arrangements became more common in tech. Now, Jay Ritter, the University of Florida professor whose work tracking and analyzing IPOs has earned him the moniker Mr. IPO, suggests that if anything, Lyft’s trajectory might make shareholders even less nervous about dual-stock structures.
For one thing, with the possible exception of Google’s founders — who came up with an entirely new share class in 2012 to preserve their power — founders lose their stranglehold on power as they sell their shares, which then convert to a one-vote-per-one-share structure. Green, for example, still controls 20% of the shareholder voting rights at Lyft, while Zimmer now controls 12% of the company’s voting rights, he told the WSJ yesterday.
Further, says Ritter, even tech companies with dual-class shares are policed by shareholders who make it clear what they will or will not tolerate. Again, just look at Lyft, whose shares were trading at 86% below their offering price earlier today in a clear sign that investors have — at least for now — lost confidence in the outfit.
We talked with Ritter last night about why stakeholders aren’t likely to push too hard against super-voting shares, despite that now would seem the time to do it. Excerpts from that conversation, below, have been lightly edited for length and clarity.
TC: Majority voting power for founders became widespread over the last dozen years or so, as VCs and even exchanges did what they could to appear founder-friendly. According to your own research, between 2012 and last year, the percentage of tech companies going public with dual-class shares shot from 15% to 46%. Should we expect this to reverse course now that the market has tightened and money isn’t flowing so freely to founders?
JR: The bargaining power of founders versus VCs has changed in the last year, that’s true, and public market investors have never been enthusiastic about founders having super voting stock. But as long as things go well, there isn’t pressure on managers to give up super voting stock. One reason U.S. investors haven’t been overly concerned about dual-class structures is that, on average, companies with dual-class structures have delivered for shareholders. It’s only when stock prices decline that people start questioning: Should we have this?
Isn’t that what we are seeing currently?
With a general downturn, even if a company is executing according to plan, shares have fallen in many cases.
So you expect that investors and public shareholders will remain complacent about this issue despite the market.
In recent years, there haven’t been a lot of examples where entrenched management is doing things wrong. There have been cases where an activist hedge fund is saying, “We don’t think you’re pursuing the right strategy.” But one of the reasons for complacency is that there are checks and balances. It’s not the case where, as in Russia, a manager can loot the company and public shareholders can’t do anything about it. They can vote with their feet. There are also shareholder lawsuits. These can be abused, but the threat of them [keeps companies in check]. What’s also true, especially of tech companies where employees have so much equity-based compensation, is that CEOs are going to be happier when their stock goes up in price but they also know their employees will be happier when the stock is doing well.
Before WeWork’s original IPO plans famously imploded in the fall of 2019, Adam Neumann expected to have so much voting control over the company that he could pass it along to future generations of Neumanns.
But when the attempt to go public backfired — [with the market saying] just because SoftBank thinks it’s worth $47 billion doesn’t mean we think it’s worth that much — he faced a trade-off. It was, “I can keep control or take a bunch of money and walk away” and “Would I rather be poorer and in control or richer and move on?” and he decided, “I’ll take the money.”
I think Lyft’s founders have the same trade-off.
Meta is perhaps a better example of a company whose CEO’s super-voting power has worried many, most recently as the company leaned into the metaverse.
A number of years ago, when Facebook was still Facebook, Mark Zuckerberg proposed doing what Larry Page and Sergey Brin had done at Google but he got a lot of pushback and backed down instead of pushing it through. Now if he wants to sell off stock to diversify his portfolio, he gives up some votes. The way most of these companies with super voting stock are structured is that if they sell it, it automatically converts into one-share-one-stock sales, so someone who buys it doesn’t get extra votes.
A story in Bloomberg earlier today asked why there are so many family dynasties in media — the Murdochs, the Sulzbergers — but not in tech. What do you think?
The media industry is different from the tech industry. Forty years ago, there was analysis of dual-class companies and, at the time, a lot of the dual-class companies were media: the [Bancroft family, which previously owned the Wall Street Journal], the Sulzbergers with the New York Times. There were also a lot of dual-class structures associated with gambling and alcohol companies before tech firms began [taking companies public with this structure in place]. But family firms are nonexistent in tech because the motivations are different; dual-class structures are [solely] meant to keep founders in control. Also, tech companies come and go pretty rapidly. With tech, you can be successful for years and then a new competitor comes along and suddenly . . .
So the bottom line, in your view, is that dual-class shares aren’t going away, no matter that shareholders don’t like them. They don’t dislike them enough to do anything about them. Is that right?
If there was concern about entrenched management pursuing stupid policies for years, investors would be demanding bigger discounts. That might have been the case with Adam Neumann; his control wasn’t something that made investors enthusiastic about the company. But for most tech companies — of which I would not consider WeWork — because you have not only the founder but employees with equity-linked compensation, there is a lot of implicit, if not explicit, pressure on shareholder value maximization rather than kowtowing to the founder’s whims. I’d be surprised if they disappeared.
Tesla brings back European referral program as end of Q1 nears
Tesla is bringing back its referral program to Europe, a strategy that taps into the brand loyalty of customers as it seeks to preserve market share and boost sales before the first quarter of 2023 closes.
The referral program follows Tesla’s move to reduce prices in a variety of markets, including Europe, China and North America.
Starting Tuesday in Europe, new Tesla buyers can receive 100 so-called “Loot Box Credits” when referred by a current Tesla owner, who will get 2,000 credits for the referral. If the referred customer takes delivery before March 31, 2023, they’ll get a bonus of 5,000 free Supercharging kilometres, and the referrer will get 10,000 credits. Those credits can be redeemed for software upgrades, up to 10,000 kilometers of free Supercharging “and more.”
Tesla has never used traditional advertising, so the company has historically used its referral program to get its loyal customer base to promote vehicles. Those rewards have changed over the last few years. At certain points, owners could win rewards like having a photo of their choosing launched into deep space orbit, an invite to an upcoming Tesla event, or even free new Roadsters to owners who accumulated enough referrals.
Tesla realized such extravagant rewards were starting to eat into profits, so in 2019 the automaker paused the program and came back with a more reasonable one that gives the referral giver and receiver 1,000 miles of free Supercharging each.
Last November, Tesla launched a revamped referral program in the U.S., which gives out credits that can be put towards the purchase of Tesla solar products, like the Solar Roof and Solar Panels. Tesla also launched a program in China called Treasure Box, where owners get credits that can be used towards the purchase of accessories like vehicle chargers, t-shirts or shot glasses.
The move in Europe suggests that Tesla is trying to hold onto, or even grow, its market share dominance. Tesla was the most popular EV brand in Europe last year, with the Model Y and Model 3 topping the ranks at 138,373 and 91,257 sales, respectively. Following behind were the Volkswagen ID.4 with 68,409 unit sales, the Fiat 500 electric with 66,732, and the Ford Kuga plug-in hybrid EV with 55,018 sales, according to Inside EVs.
While Tesla was the most popular EV brand in Europe last year, it actually falls behind the large multi-brand OEMs. Volkswagen Group, which includes brands like Audi and VW, actually has the largest market share of plug-in EVs with 20.6%. Stellantis, BMW Group and Hyundai follow with 14.6%, 10.5% and 10.1%, respectively. Mercedes and Tesla are tied at around 9% share.
As of this week, Tesla has finally hit production capacity of 5,000 vehicles per week at its Berlin gigafactory — a milestone CEO Elon Musk had originally promised for the end of 2022. While production numbers don’t equal sales, it’s possible that the increased production in Europe could help the automaker maintain its position and gain even more market share in the future.
The referral program isn’t the only move Tesla has made to boost sales, particularly before it reports quarterly earnings. In January, Tesla cut prices for Model 3 and Model Y vehicles in the U.S. and Europe by 20%. Earlier this month, the automaker slashed Model S and Model X prices in the U.S. as well.
In December 2022, Tesla also provided up to $7,500 discounts for vehicles purchased and delivered before the end of the year in the hopes of attracting buyers who might otherwise wait for the new year when Inflation Reduction Act incentives would kick in.
Pinterest brings shopping capabilities to Shuffles, its collage-making app
Pinterest announced today that it’s testing ways to integrate Shuffles collage content into Pinterest, starting with shopping. Shuffles, which is Pinterest’s collage-making app, launched to general public last November. To use Shuffles, users build collages using Pinterest’s own photo library or by snapping photos of objects they want to include with their iPhone’s camera. The iOS-only app is available in the U.S., Canada, Great Britain, Ireland, Australia and New Zealand.
Shuffles will now have all of the shopping capabilities as regular pins. Users will be able to tap individual cutouts used in collages, see the brand, price, and other product metadata along with similar products to shop.
“Unlike typical product exploration, Shuffles bring an interactivity that makes the experience inspirational and fun,” the company said in a blog post. “Gen-Z is curating fresh, relevant content alongside their peers, which is quickly making for a marketplace of trendy, shoppable ideas. The high density nature of Shuffles, which can include layers of product cutouts from multiple Pins, allows consumers to dig deeper and also connect to other Shuffles that include the same Pins. As we look ahead to how consumer behavior is evolving, we’re testing ways of integrating Shuffles collage content into Pinterest, starting with shopping.”
Although Shuffles surged to become the No. 1 Lifestyle app on the U.S. App Store in August when it was invite-only, the app’s popularity has since declined. By bringing shopping capabilities to Shuffles, Pinterest is likely looking for ways to retain users on the standalone app.
Pinterest also announced that it’s exploring a new takeover feature for advertisers called “Pinterest Premiere Spotlight” that prominently showcases a brand on search. The company says the feature is designed give advertisers a new way to reach users on Pinterest.
The company says 97% of top searches on Pinterest are unbranded, which means users typically don’t type a brand name into their searches on the platform. This gives brands the opportunity to be discovered as they help consumers go from discovery to decision to purchase, Pinterest says. In the coming months, the company planes to offer additional ways to help brands connect with shoppers.
Pinterest also shared some new stats about its Catalogs offering, which lets brands upload their full catalog to the platform and turn their products into dynamic Product Pins. The company says it has seen a 66% increase in retailers setting up shop by uploading or integrating their digital catalogs on its platform, along with 70% growth in active shopping feeds year over year globally.
As part of its most recent earnings release, Pinterest revealed that its platform now has 450 million monthly active users globally, a 4% jump year-on-year. Pinterest has been focused on enhancing the shopping experience on its platform over the past few years, and said during its earnings call that it wants to make every pin shoppable, including videos.
Interviews1 year ago
Interview with Jean-Francois Desormeaux, Real Estate Investor
Business News10 months ago
NFTMagazine.com Is Bringing NFTMag Conference 2022 to Miami this Year Says JetSetFly
Technology6 months ago
General Atlantic buys out SoftBank’s 15% stake in edtech Kahoot, now valued at about $152M vs the $215M SoftBank ponied up 2 years ago
Interviews12 months ago
Paying it Forward — Meet Dr. Jonathan Kenigson, the Founder of the World’s Leading Think-Tank in the Quadrivium
Interviews4 months ago
Interview with Justice Mitchell, A 16-year-old Student-Athlete Who Received a Basketball Scholarship Offer from Pennsylvania University Greater Allegheny
Entrepreneurship1 year ago
600% In Under 5 Years, Financial Advisors Grow Business By Podcasting And YouTube
Entrepreneurship1 year ago
Muminovic Benjamin E-commerce on Shopify the Course of the Business Man
Community9 months ago
The Bassnectar Community – It Belongs to All of Us