#[1]AP News IFRAME: [2]https://www.googletagmanager.com/ns.html?id=GTM-MCLSCF8 AP NEWS Email: Get AP News stories (BUTTON) Go Listen (BUTTON) Sections * [3]U.S. News * [4]World News * [5]Politics * [6]Sports * [7]Entertainment * [8]Business * [9]Technology * [10]Health * [11]Science * [12]Oddities * [13]Lifestyle * [14]Photography * [15]Videos Listen (BUTTON) Sections 1. [16]AP Top News 2. [17]U.S. News 3. [18]World News[19]Latest on Russia-Ukraine war[20]Africa[21]Asia Pacific[22]Australia[23]Europe[24]Latin America[25]Middle East 4. [26]Politics[27]President Biden[28]Congress[29]Supreme Court[30]Election 2023 5. [31]Sports[32]MLB[33]NBA playoffs[34]NHL[35]NFL[36]Tennis[37]Golf 6. [38]Entertainment[39]Film reviews[40]Movies[41]Music[42]Television[43]Fashion 7. [44]Business[45]U.S. economy[46]Financial markets ______________________________________________________________ 8. [47]Videos 9. [48]Technology 10. [49]Health[50]COVID-19 11. More[51]AP Investigations[52]Climate and environment[53]Oddities[54]Photography[55]Travel[56]Science[57]AP Fact Check[58]Lifestyle[59]Religion[60]Press Releases (BUTTON) * [61]George Santos charges * [62]Trump rape trial verdict * [63]Pulitzer winning Mariupol coverage * [64]Texas mall shooting * [65]Latest on Russia-Ukraine war * [66]NBA Playoffs ____________________ (BUTTON) Search https://apnews.com/article/hacking-jailbreaking-chatgpt-bing ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ Click to copy https://apnews.com/article/hacking-jailbreaking-chatgpt-bing ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ Click to copy Related topics * [67]Technology * [68]Science * [69]Artificial intelligence * [70]AP Top News * [71]Business Mass event will let hackers test limits of AI technology By MATT O'BRIENMay 10, 2023 GMT 1 of 4 Rumman Chowdhury, co-founder of Humane Intelligence, a nonprofit developing accountable AI systems, poses for a photograph at her home Monday, May 8, 2023, in Katy, Texas. ChatGPT maker OpenAI, and other major AI providers such as Google and Microsoft, are coordinating with the Biden administration to let thousands of hackers take a shot at testing the limits of their technology. Chowdhury is the lead coordinator of the mass hacking event planned for this summer's DEF CON hacker convention in Las Vegas. (AP Photo/David J. Phillip) 1 of 4 Rumman Chowdhury, co-founder of Humane Intelligence, a nonprofit developing accountable AI systems, poses for a photograph at her home Monday, May 8, 2023, in Katy, Texas. ChatGPT maker OpenAI, and other major AI providers such as Google and Microsoft, are coordinating with the Biden administration to let thousands of hackers take a shot at testing the limits of their technology. Chowdhury is the lead coordinator of the mass hacking event planned for this summer's DEF CON hacker convention in Las Vegas. (AP Photo/David J. Phillip) No sooner did ChatGPT get unleashed than hackers started “jailbreaking” the artificial intelligence chatbot — trying to override its safeguards so it could blurt out something unhinged or obscene. But now its maker, OpenAI, and other major AI providers such as Google and Microsoft, are [72]coordinating with the Biden administration to let thousands of hackers take a shot at testing the limits of their technology. Some of the things they’ll be looking to find: How can chatbots be manipulated to cause harm? Will they share the private information we confide in them to other users? And why do they assume a doctor is a man and a nurse is a woman? “This is why we need thousands of people,” said Rumman Chowdhury, a coordinator of the mass hacking event planned for this summer’s DEF CON hacker convention in Las Vegas that’s expected to draw several thousand people. “We need a lot of people with a wide range of lived experiences, subject matter expertise and backgrounds hacking at these models and trying to find problems that can then go be fixed.” Anyone who’s tried ChatGPT, Microsoft’s Bing chatbot or Google’s Bard will have quickly learned that they have a tendency [73]to fabricate information and confidently present it as fact. These systems, [74]built on what’s known as large language models, also emulate the cultural biases they’ve learned from being trained upon huge troves of what people have written online. The idea of a mass hack caught the attention of U.S. government officials in March at the South by Southwest festival in Austin, Texas, where Sven Cattell, founder of DEF CON’s long-running AI Village, and Austin Carson, president of responsible AI nonprofit SeedAI, helped lead a workshop inviting community college students to hack an AI model. Carson said those conversations eventually blossomed into a proposal to test AI language models following the guidelines of [75]the White House’s Blueprint for an AI Bill of Rights — a set of principles to limit the impacts of algorithmic bias, [76]give users control over their data and ensure that automated systems are used safely and transparently. There’s already a community of users trying their best to trick chatbots and highlight their flaws. Some are official “red teams” authorized by the companies to “prompt attack” the AI models to discover their vulnerabilities. Many others are hobbyists showing off humorous or disturbing outputs on social media until they get banned for violating a product’s terms of service. “What happens now is kind of a scattershot approach where people find stuff, it goes viral on Twitter,” and then it may or may not get fixed if it’s egregious enough or the person calling attention to it is influential, Chowdhury said. In one example, known as the “grandma exploit,” users were able to get chatbots to tell them how to make a bomb — a request a commercial chatbot would normally decline — by asking it to pretend it was a grandmother telling a bedtime story about how to make a bomb. In another example, searching for Chowdhury using [77]an early version of Microsoft’s Bing search engine chatbot — which is based on the same technology as ChatGPT but can pull real-time information from the internet — led to a profile that speculated Chowdhury “loves to buy new shoes every month” and made strange and gendered assertions about her physical appearance. Chowdhury helped introduce a method for rewarding the discovery of algorithmic bias to DEF CON’s AI Village in 2021 when she was the head of Twitter’s AI ethics team — a job that has since been eliminated upon Elon Musk’s October takeover of the company. Paying hackers a “bounty” if they uncover a security bug is commonplace in the cybersecurity industry — but it was a newer concept to researchers studying harmful AI bias. This year’s event will be at a much greater scale, and is the first to tackle the large language models that have attracted a surge of public interest and commercial investment since the release of ChatGPT late last year. Chowdhury, now the co-founder of AI accountability nonprofit Humane Intelligence, said it’s not just about finding flaws but about figuring out ways to fix them. “This is a direct pipeline to give feedback to companies,” she said. “It’s not like we’re just doing this hackathon and everybody’s going home. We’re going to be spending months after the exercise compiling a report, explaining common vulnerabilities, things that came up, patterns we saw.” [78] Artificial Intelligence In global rush to regulate AI, Europe set to be trailblazer Could AI pen 'Casablanca'? Screenwriters take aim at ChatGPT Screenwriters take aim at artificial intelligence, ChatGPT Biden, Harris meet with CEOs about AI risks Some of the details are still being negotiated, but companies that have agreed to provide their models for testing include OpenAI, Google, chipmaker Nvidia and startups Anthropic, Hugging Face and Stability AI. Building the platform for the testing is another startup called Scale AI, known for its work in assigning humans to [79]help train AI models by labeling data. “As these foundation models become more and more widespread, it’s really critical that we do everything we can to ensure their safety,” said Scale CEO Alexandr Wang. “You can imagine somebody on one side of the world asking it some very sensitive or detailed questions, including some of their personal information. You don’t want any of that information leaking to any other user.” Other dangers Wang worries about are chatbots that give out “unbelievably bad medical advice” or other misinformation that can cause serious harm. Anthropic co-founder Jack Clark said the DEF CON event will hopefully be the start of a deeper commitment from AI developers to measure and evaluate the safety of the systems they are building. “Our basic view is that AI systems will need third-party assessments, both before deployment and after deployment. Red-teaming is one way that you can do that,” Clark said. “We need to get practice at figuring out how to do this. It hasn’t really been done before.” AP NEWS 1. [80]Top Stories 2. [81]Video 3. [82]Contact Us 4. [83]Accessibility Statement 5. (BUTTON) Cookie Settings Download AP NEWS Connect with the definitive source for global and local news More from AP 1. [84]ap.org 2. [85]AP Insights 3. [86]AP Definitive Source Blog 4. [87]AP Images Spotlight 5. [88]AP Explore 6. [89]AP Books 7. [90]AP Stylebook Follow AP 1. 2. 3. 4. The Associated Press 1. [91]About 2. [92]Contact 3. [93]Customer Support 4. [94]Careers 5. [95]Terms & Conditions 6. [96]Privacy All contents © copyright 2023 The Associated Press. All rights reserved. References Visible links 1. https://apnews.com/OpenSearchDescription.xml 2. https://www.googletagmanager.com/ns.html?id=GTM-MCLSCF8 3. https://apnews.com/hub/us-news?utm_source=apnewsnav&utm_medium=navigation 4. https://apnews.com/hub/world-news?utm_source=apnewsnav&utm_medium=navigation 5. https://apnews.com/hub/politics?utm_source=apnewsnav&utm_medium=navigation 6. https://apnews.com/hub/sports?utm_source=apnewsnav&utm_medium=navigation 7. https://apnews.com/hub/entertainment?utm_source=apnewsnav&utm_medium=navigation 8. https://apnews.com/hub/business?utm_source=apnewsnav&utm_medium=navigation 9. https://apnews.com/hub/technology?utm_source=apnewsnav&utm_medium=navigation 10. https://apnews.com/hub/health?utm_source=apnewsnav&utm_medium=navigation 11. https://apnews.com/hub/science?utm_source=apnewsnav&utm_medium=navigation 12. https://apnews.com/hub/oddities?utm_source=apnewsnav&utm_medium=navigation 13. https://apnews.com/hub/lifestyle?utm_source=apnewsnav&utm_medium=navigation 14. https://apnews.com/hub/photography?utm_source=apnewsnav&utm_medium=navigation 15. https://apnews.com/hub/videos?utm_source=apnewsnav&utm_medium=navigation 16. https://apnews.com/hub/ap-top-news?utm_source=apnewsnav&utm_medium=sections 17. https://apnews.com/hub/us-news?utm_source=apnewsnav&utm_medium=sections 18. https://apnews.com/hub/world-news?utm_source=apnewsnav&utm_medium=sections 19. https://apnews.com/hub/russia-ukraine?utm_source=apnewsnav&utm_medium=sections 20. https://apnews.com/hub/africa?utm_source=apnewsnav&utm_medium=sections 21. https://apnews.com/hub/asia-pacific?utm_source=apnewsnav&utm_medium=sections 22. https://apnews.com/hub/australia?utm_source=apnewsnav&utm_medium=sections 23. https://apnews.com/hub/europe?utm_source=apnewsnav&utm_medium=sections 24. https://apnews.com/hub/latin-america?utm_source=apnewsnav&utm_medium=sections 25. https://apnews.com/hub/middle-east?utm_source=apnewsnav&utm_medium=sections 26. https://apnews.com/hub/politics?utm_source=apnewsnav&utm_medium=sections 27. https://apnews.com/hub/joe-biden?utm_source=apnewsnav&utm_medium=sections 28. https://apnews.com/hub/united-states-congress?utm_source=apnewsnav&utm_medium=sections 29. https://apnews.com/hub/us-supreme-court?utm_source=apnewsnav&utm_medium=sections 30. https://apnews.com/hub/election-2023?utm_source=apnewsnav&utm_medium=sections 31. https://apnews.com/hub/sports?utm_source=apnewsnav&utm_medium=sections 32. https://apnews.com/hub/mlb?utm_source=apnewsnav&utm_medium=sections 33. https://apnews.com/hub/nba?utm_source=apnewsnav&utm_medium=sections 34. https://apnews.com/hub/nhl?utm_source=apnewsnav&utm_medium=sections 35. https://apnews.com/hub/nfl?utm_source=apnewsnav&utm_medium=sections 36. https://apnews.com/hub/tennis?utm_source=apnewsnav&utm_medium=sections 37. https://apnews.com/hub/golf?utm_source=apnewsnav&utm_medium=sections 38. https://apnews.com/hub/entertainment?utm_source=apnewsnav&utm_medium=sections 39. https://apnews.com/hub/film-reviews?utm_source=apnewsnav&utm_medium=sections 40. https://apnews.com/hub/movies?utm_source=apnewsnav&utm_medium=sections 41. https://apnews.com/hub/music?utm_source=apnewsnav&utm_medium=sections 42. https://apnews.com/hub/television?utm_source=apnewsnav&utm_medium=sections 43. https://apnews.com/hub/fashion?utm_source=apnewsnav&utm_medium=sections 44. https://apnews.com/hub/business?utm_source=apnewsnav&utm_medium=sections 45. https://apnews.com/hub/economy?utm_source=apnewsnav&utm_medium=sections 46. https://apnews.com/hub/financial-markets?utm_source=apnewsnav&utm_medium=sections 47. https://apnews.com/hub/videos?utm_source=apnewsnav&utm_medium=sections 48. https://apnews.com/hub/technology?utm_source=apnewsnav&utm_medium=sections 49. https://apnews.com/hub/health?utm_source=apnewsnav&utm_medium=sections 50. https://apnews.com/hub/coronavirus-pandemic?utm_source=apnewsnav&utm_medium=sections 51. https://apnews.com/hub/ap-investigations?utm_source=apnewsnav&utm_medium=sections 52. https://apnews.com/hub/climate-and-environment?utm_source=apnewsnav&utm_medium=sections 53. https://apnews.com/hub/oddities?utm_source=apnewsnav&utm_medium=sections 54. https://apnews.com/hub/photography?utm_source=apnewsnav&utm_medium=sections 55. https://apnews.com/hub/travel?utm_source=apnewsnav&utm_medium=sections 56. https://apnews.com/hub/science?utm_source=apnewsnav&utm_medium=sections 57. https://apnews.com/hub/ap-fact-check?utm_source=apnewsnav&utm_medium=sections 58. https://apnews.com/hub/lifestyle?utm_source=apnewsnav&utm_medium=sections 59. https://apnews.com/hub/religion?utm_source=apnewsnav&utm_medium=sections 60. https://apnews.com/hub/press-releases?utm_source=apnewsnav&utm_medium=sections 61. https://apnews.com/article/george-santos-justice-department-new-york-7e16d39eea0fc577f78d17502a384084?utm_source=apnewsnav&utm_medium=featured 62. https://apnews.com/article/trump-rape-carroll-trial-fe68259a4b98bb3947d42af9ec83d7db?utm_source=apnewsnav&utm_medium=featured 63. https://apnews.com/article/ap-pulitzers-mariupol-russia-d4bde22e1caf44ec4663b58723c403b7?utm_source=apnewsnav&utm_medium=featured 64. https://apnews.com/article/shooting-outlet-mall-allen-texas-200f1ffadf7daefa42cfbe45510b083f?utm_source=apnewsnav&utm_medium=featured 65. https://apnews.com/hub/russia-ukraine?utm_source=apnewsnav&utm_medium=featured 66. https://apnews.com/hub/nba-playoffs?utm_source=apnewsnav&utm_medium=featured 67. https://apnews.com/hub/technology 68. https://apnews.com/hub/science 69. https://apnews.com/hub/artificial-intelligence 70. https://apnews.com/hub/ap-top-news 71. https://apnews.com/hub/business 72. https://apnews.com/article/ai-artificial-intelligence-white-house-harris-578d623e473b0eeb3fa3e4728d7e9868 73. https://apnews.com/article/kansas-city-chiefs-philadelphia-eagles-technology-science-82bc20f207e3e4cf81abc6a5d9e6b23a 74. https://apnews.com/hub/artificial-intelligence 75. https://apnews.com/article/technology-business-artificial-intelligence-7a39848340d210592aeea2478225f489 76. https://apnews.com/article/chatgpt-openai-data-privacy-italy-b9ab3d12f2b2cfe493237fd2b9675e21 77. https://apnews.com/article/technology-science-microsoft-corp-business-software-dd445694f34a6b7a0444db9988330229 78. https://apnews.com/hub/artificial-intelligence 79. https://apnews.com/article/north-america-india-us-news-ap-top-news-venezuela-1f58465e55d643ea84e51713f35ad214 80. https://apnews.com/hub/ap-top-news 81. https://apnews.com/hub/videos 82. mailto:support@apnews.com 83. https://apnews.com/accessibility-statement 84. https://www.ap.org/ 85. https://insights.ap.org/ 86. https://blog.ap.org/ 87. https://apimagesblog.com/ 88. https://www.ap.org/explore/ 89. https://www.ap.org/books/ 90. https://www.apstylebook.com/ 91. https://www.ap.org/about/ 92. https://www.ap.org/contact-us/ 93. http://aphelp.ap.org/ 94. https://www.ap.org/careers/ 95. https://apnews.com/termsofservice 96. https://apnews.com/privacystatement Hidden links: 98. https://apnews.com/ 99. https://facebook.com/dialog/share?app_id=870613919693099&display=popup&href=https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f 100. https://twitter.com/intent/tweet?url=https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f 101. mailto:?subject=Mass%20event%20will%20let%20hackers%20test%20limits%20of%20AI%20technology&body=https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f 102. https://facebook.com/dialog/share?app_id=870613919693099&display=popup&href=https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f 103. https://twitter.com/intent/tweet?url=https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f 104. mailto:?subject=Mass%20event%20will%20let%20hackers%20test%20limits%20of%20AI%20technology&body=https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f 105. https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f/gallery/2d174409b10c4ff69d251643868bac80 106. https://apnews.com/article/hacking-jailbreaking-chatgpt-bing-defcon-biden-ai-97b963db084800f11b26b8a023b1713f/gallery/2d174409b10c4ff69d251643868bac80 107. https://apnews.com/article/artificial-intelligence-chatgpt-europe-rules-906fc89d2561b200fa6eb40a06b946a5 108. https://apnews.com/article/ai-hollywood-writers-strike-artificial-intelligence-dc71c4cabcca0ee1b1afe0050d392a36 109. https://apnews.com/video/entertainment-associated-press-artificial-intelligence-ba42e8d70a4a48e989ae98269c80d741 110. https://apnews.com/article/ai-artificial-intelligence-white-house-harris-578d623e473b0eeb3fa3e4728d7e9868 111. https://twitter.com/AP 112. https://www.facebook.com/APNews 113. https://www.youtube.com/user/AssociatedPress 114. https://www.linkedin.com/company/associated-press