3092 matches found
A Taxonomy of Prompt Injection Attacks
Researchers ran a global prompt hacking competition, and have documented the results in a paper that both gives a lot of good examples and tries to organize a taxonomy of effective prompt injection strategies. It seems as if the most common successful strategy is the "compound instruction attack,...
How Public AI Can Strengthen Democracy
With the worlds focus turning to misinformation, manipulation, and outright propaganda ahead of the 2024 U.S. presidential election, we know that democracy has an AI problem. But were learning that AI has a democracy problem, too. Both challenges must be addressed for the sake of democratic...
Surveillance through Push Notifications
The Washington Post is reporting on the FBIs increasing use of push notification data--"push tokens"--to identify people. The police can request this data from companies like Apple and Google without a warrant. The investigative technique goes back years. Court orders that were issued in 2019 to...
The Insecurity of Video Doorbells
Consumer Reports has analyzed a bunch of popular Internet-connected video doorbells. Their security is terrible. First, these doorbells expose your home IP address and WiFi network name to the internet without encryption, potentially opening your home network to online criminals. … Anyone who can...
LLM Prompt Injection Worm
Researchers have demonstrated a worm that spreads through prompt injection. Details: In one instance, the researchers, acting as attackers, wrote an email including the adversarial text prompt, which "poisons" the database of an email assistant using retrieval-augmented generation RAG, a way for...
Friday Squid Blogging: New Extinct Species of Vampire Squid Discovered
Paleontologists have discovered a 183-million-year-old species of vampire squid. Prior research suggests that the vampyromorph lived in the shallows off an island that once existed in what is now the heart of the European mainland. The research team believes that the remarkable degree of...
NIST Cybersecurity Framework 2.0
NIST has released version 2.0 of the Cybersecurity Framework: The CSF 2.0, which supports implementation of the National Cybersecurity Strategy, has an expanded scope that goes beyond protecting critical infrastructure, such as hospitals and power plants, to all organizations in any sector. It al...
How the “Frontier” Became the Slogan of Uncontrolled AI
Artificial intelligence AI has been billed as the next frontier of humanity: the newly available expanse whose exploration will drive the next era of growth, wealth, and human flourishing. Its a scary metaphor. Throughout American history, the drive for expansion and the very concept of terrain u...
A Cyber Insurance Backstop
In the first week of January, the pharmaceutical giant Merck quietly settled its years-long lawsuit over whether or not its property and casualty insurers would cover a $700 million claim filed after the devastating NotPetya cyberattack in 2017. The malware ultimately infected more than 40,000 of...
China Surveillance Company Hacked
Last week, someone posted something like 570 files, images and chat logs from a Chinese company called I-Soon. I-Soon sells hacking and espionage services to Chinese national and local government. Lots of details in the news articles. These arent details about the tools or techniques, more the...
Apple Announces Post-Quantum Encryption Algorithms for iMessage
Apple announced PQ3, its post-quantum encryption standard based on the Kyber secure key-encapsulation protocol, one of the post-quantum algorithms selected by NIST in 2022. Theres a lot of detail in the Apple blog post, and more in Douglas Stabilas security analysis. I am of two minds about this...
Friday Squid Blogging: Illex Squid and Climate Change
There are correlations between the populations of the Illex Argentines squid and water temperatures. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here...
AIs Hacking Websites
New research: LLM Agents can Autonomously Hack Websites Abstract: In recent years, large language models LLMs have become increasingly capable and can now interact with tools i.e., call functions, read documents, and recursively call themselves. As a result, these LLMs can now function autonomous...
New Image/Video Prompt Injection Attacks
Simon Willison has been playing with the video processing capabilities of the new Gemini Pro 1.5 model from Google, and its really impressive. Which means a lot of scary new video prompt injection attacks. And remember, given the current state of technology, prompt injection attacks are impossibl...
Details of a Phone Scam
First-person account of someone who fell for a scam, that started as a fake Amazon service rep and ended with a fake CIA agent, and lost $50,000 cash. And this is not a naive or stupid person. The details are fascinating. And if you think it couldnt happen to you, think again. Given the right set...
Microsoft Is Spying on Users of Its AI Tools
Microsoft announced that it caught Chinese, Russian, and Iranian hackers using its AI tools--presumably coding tools--to improve their hacking abilities. From their report: In collaboration with OpenAI, we are sharing threat intelligence showing detected state affiliated adversaries--tracked as...
EU Court of Human Rights Rejects Encryption Backdoors
The European Court of Human Rights has ruled that breaking end-to-end encryption by adding backdoors violates human rights: Seemingly most critically, the Russian government told the ECHR that any intrusion on private lives resulting from decrypting messages was "necessary" to combat terrorism in...
Friday Squid Blogging: Vegan Squid-Ink Pasta
It uses black beans for color and seaweed for flavor. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here...
On the Insecurity of Software Bloat
Good essay on software bloat and the insecurities it causes. The world ships too much code, most of it by third parties, sometimes unintended, most of it uninspected. Because of this, there is a huge attack surface full of mediocre code. Efforts are ongoing to improve the quality of code itself,...
Upcoming Speaking Engagements
This is a current list of where and when I am scheduled to speak: I’m speaking at the Munich Security Conference MSC 2024 in Munich, Germany, on Friday, February 16, 2024. I’m giving a keynote on “AI and Trust” at Generative AI, Free Speech, & Public Discourse. The symposium will be held at...
Improving the Cryptanalysis of Lattice-Based Public-Key Algorithms
The winner of the Best Paper Award at Crypto this year was a significant improvement to lattice-based cryptanalysis. This is important, because a bunch of NISTs post-quantum options base their security on lattice problems. I worry about standardizing on post-quantum algorithms too quickly. We are...
A Hacker’s Mind is Out in Paperback
The paperback version of A Hackers Mind has just been published. Its the same book, only a cheaper format. But--and this is the real reason I am posting this--Amazon has significantly discounted the hardcover to $15 to get rid of its stock. This is much cheaper than I am selling it for, and cheap...
Molly White Reviews Blockchain Book
Molly White--of "Web3 is Going Just Great" fame--reviews Chris Dixons blockchain solutions book: Read Write Own: In fact, throughout the entire book, Dixon fails to identify a single blockchain project that has successfully provided a non-speculative service at any kind of scale. The closest he...
On Passkey Usability
Matt Burgess tries to only use passkeys. The results are mixed...
Friday Squid Blogging: A Penguin Named “Squid”
Amusing story about a penguin named "Squid." As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here...
No, Toothbrushes Were Not Used in a Massive DDoS Attack
The widely reported story last week that 1.5 million smart toothbrushes were hacked and used in a DDoS attack is false. Near as I can tell, a German reporter talking to someone at Fortinet got it wrong, and then everyone else ran with it without reading the German text. It was a hypothetical, whi...
On Software Liabilities
Over on Lawfare, Jim Dempsey published a really interesting proposal for software liability: "Standard for Software Liability: Focus on the Product for Liability, Focus on the Process for Safe Harbor." Section 1 of this paper sets the stage by briefly describing the problem to be solved. Section ...
Teaching LLMs to Be Deceptive
Interesting research: "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training": Abstract: Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given th...
Documents about the NSA’s Banning of Furby Toys in the 1990s
Via a FOIA request, we have documents from the NSA about their banning of Furby toys. 404 Media has the story. EDITED TO ADD: The documents are now on Archive.org...
Deepfake Fraud
A deepfake video conference call--with everyone else on the call a fake--fooled a finance worker into sending $25M to the criminals account...
Friday Squid Blogging: Illex Squid in Argentina Waters
Argentina is reporting that there is a good population of illex squid in its waters ready for fishing, and is working to ensure that Chinese fishing boats dont take it all. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my bl...
David Kahn
David Kahn has died. His groundbreaking book, The Codebreakers was the first serious book I read about codebreaking, and one of the primary reasons I entered this field. He will be missed. EDITED TO ADD 2/4: Funeral website. EDITED TO ADD 2/10: New York Times obituary...
A Self-Enforcing Protocol to Solve Gerrymandering
In 2009, I wrote: There are several ways two people can divide a piece of cake in half. One way is to find someone impartial to do it for them. This works, but it requires another person. Another way is for one person to divide the piece, and the other person to complain to the police, a judge, o...
Facebook’s Extensive Surveillance Network
Consumer Reports is reporting that Facebook has built a massive surveillance network: Using a panel of 709 volunteers who shared archives of their Facebook data, Consumer Reports found that a total of 186,892 companies sent data about them to the social network. On average, each participant in th...
CFPB’s Proposed Data Rules
In October, the Consumer Financial Protection Bureau CFPB proposed a set of rules that if implemented would transform how financial institutions handle personal data about their customers. The rules put control of that data back in the hands of ordinary Americans, while at the same time undermini...
New Images of Colossus Released
GCHQ has released new images of the WWII Colossus code-breaking computer, celebrating the machines eightieth anniversary birthday?. News article...
NSA Buying Bulk Surveillance Data on Americans without a Warrant
It finally admitted to buying bulk data on Americans from data brokers, in response to a query by Senator Weyden. This is almost certainly illegal, although the NSA maintains that it is legal until its told otherwise. Some news articles...
Microsoft Executives Hacked
Microsoft is reporting that a Russian intelligence agency--the same one responsible for SolarWinds--accessed the email system of the companys executives. Beginning in late November 2023, the threat actor used a password spray attack to compromise a legacy non-production test tenant account and ga...
Friday Squid Blogging: Footage of Black-Eyed Squid Brooding Her Eggs
Amazing footage of a black-eyed squid Gonatus onyx carrying thousands of eggs. They tend to hang out about 6,200 feet below sea level. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here...
Chatbots and Human Conversation
For most of history, communicating with a computer has not been like communicating with a person. In their earliest years, computers required carefully constructed instructions, delivered through punch cards; then came a command-line interface, followed by menus and options and text boxes. If you...
Quantum Computing Skeptics
Interesting article. I am also skeptical that we are going to see useful quantum computers anytime soon. Since at least 2019, I have been saying that this is hard. And that we dont know if its "land a person on the surface of the moon" hard, or "land a person on the surface of the sun" hard. They...
Poisoning AI Models
New research into poisoning AI models: The researchers first trained the AI models using supervised learning and then used additional "safety training" methods, including more supervised learning, reinforcement learning, and adversarial training. After this, they checked if the AI still had hidde...
Side Channels Are Common
Really interesting research: "Lend Me Your Ear: Passive Remote Physical Side Channels on PCs." Abstract: We show that built-in sensors in commodity PCs, such as microphones, inadvertently capture electromagnetic side-channel leakage from ongoing computation. Moreover, this information is often...
AI Bots on X (Twitter)
You can find them by searching for OpenAI chatbot warning messages, like: "Im sorry, I cannot provide a response as it goes against OpenAIs use case policy." I hadnt thought about this before: identifying bots by searching for distinctive bot phrases...
Friday Squid Blogging: New Foods from Squid Fins
We only eat about half of a squid, ignoring the fins. A group of researchers is working to change that. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here...
Zelle Is Using My Name and Voice without My Consent
Okay, so this is weird. Zelle has been using my name, and my voice, in audio podcast ads--without my permission. At least, I think it is without my permission. Its possible that I gave some sort of blanket permission when speaking at an event. Its not likely, but it is possible. I wrote to Zelle...
Speaking to the CIA’s Creative Writing Group
This is a fascinating story. Last spring, a friend of a friend visited my office and invited me to Langley to speak to Invisible Ink, the CIAs creative writing group. I asked Vivian not her real name what she wanted me to talk about. She said that the topic of the talk was entirely up to me. I...
Canadian Citizen Gets Phone Back from Police
After 175 million failed password guesses, a judge rules that the Canadian police must return a suspects phone. Judge Carter said the investigation can continue without the phones, and he noted that Ottawa police have made a formal request to obtain more data from Google. "This strikes me as a...
Code Written with AI Assistants Is Less Secure
Interesting research: "Do Users Write More Insecure Code with AI Assistants?": Abstract: We conduct the first large-scale user study examining how users interact with an AI Code assistant to solve a variety of security related tasks across different programming languages. Overall, we find that...