‘Please write this code’ and other problematic prompts for chatbots
itbrew.com · 535 words · saved by 1 readers
When malicious prompts are answered ‘like magic,’ services using generative AI must be watched closely.
David Maynor, a security professional for over 20 years, nearly fell out of his chair when the now-practically-famous ChatGPT artificial intelligence bot gave him exactly what he asked for: Please write a python3-based c2 framework. In about 30 seconds, ChatGPT unfurled a Python 3-coded command-and-control (“C2”) server, a setup often used by hackers to control a network of machines. “I’ve been doing this for 22 years, right? This was like magic to me,” said Maynor, head of the threat intelligence group at Cybrary, a cybersecurity training platform. Generative, text-based AI and ChatGPT…
saved by
related reading
- Brown CS: Brown CS Blogblog.cs.brown.edu
- ChatGPT Is Dumber Than You Think - The Atlantictheatlantic.com
- Doomprompting Is the New Doomscrolling.workingtheorys.com
- gpt-4-system-card.pdfcdn.openai.com
- Security incident disclosure — July 2026huggingface.co
- Collections: On ChatGPT – A Collection of Unmitigated Pedantryacoup.blog
- AI #77: A Few Upgrades - by Zvi Mowshowitzthezvi.substack.com
- Attention is All You Neednotboring.co
- SoundBytes - ScaleUp:AIscaleup.events
- I used AI. It worked. I hated it.: Taggart Techtaggart-tech.com
- What should you use ChatGPT for? | ✰Vicki Boykis✰vickiboykis.com
- A guide to prompting AI (for what it is worth)oneusefulthing.org