Evaluating Real-World Adversarial ML Attack Risks and Effective Management: Robustness vs Non-ML Mitigations

The MLSecOps Podcast

Recon: Automated Red Teaming for GenAI

27:52

We made it— 300 episodes of This Is Woman’s Work ! And we’re marking this milestone by giving you something that could seriously change the game in your business or career: the skill of pitching yourself effectively. Whether you’re dreaming of being a podcast guest, landing a speaking gig, signing a client, or just asking for what you want with confidence—you’re already pitching yourself, every day. But are you doing it well? In this milestone episode, Nicole breaks down exactly how to pitch yourself to be a podcast guest … and actually hear “yes.” With hundreds of pitches landing in her inbox each month, she shares what makes a guest stand out (or get deleted), the biggest mistakes people make, and why podcast guesting is still one of the most powerful ways to grow your reach, authority, and influence. In This Episode, We Cover: ✅ Why we all need to pitch ourselves—and how to do it without feeling gross ✅ The step-by-step process for landing guest spots on podcasts (and more) ✅ A breakdown of the 3 podcast levels: Practice, Peer, and A-List—and how to approach each ✅ The must-haves of a successful podcast pitch (including real examples) ✅ How to craft a pitch that gets read, gets remembered, and gets results Whether you’re new to pitching or want to level up your game, this episode gives you the exact strategy Nicole and her team use to land guest spots on dozens of podcasts every year. Because your voice deserves to be heard. And the world needs what only you can bring. 🎁 Get the FREE Podcast Pitch Checklist + Additional Information on your Practice Group, Peer Group, and A-List Group Strategies: https://nicolekalil.com/podcast 📥 Download The Podcast Pitch Checklist Here Related Podcast Episodes: Shameless and Strategic: How to Brag About Yourself with Tiffany Houser | 298 How To Write & Publish A Book with Michelle Savage | 279 How To Land Your TED Talk and Skyrocket Your Personal Brand with Ashley Stahl | 250 Share the Love: If you found this episode insightful, please share it with a friend, tag us on social media, and leave a review on your favorite podcast platform! 🔗 Subscribe & Review: Apple Podcasts | Spotify | Amazon Music…

hace un año 41:19

MP3•Episodio en casa

Send us a text

In this episode, co-hosts Badar Ahmed and Daryan Dehghanpisheh are joined by Drew Farris (Principal, Booz Allen Hamilton) and Edward Raff (Chief Scientist, Booz Allen Hamilton) to discuss themes from their paper, "You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks," co-authored with Michael Benaroch.

Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com.
Additional tools and resources to check out:
Protect AI Guardian: Zero Trust for ML Models

Protect AI’s ML Security-Focused Open Source Tools

LLM Guard Open Source Security Toolkit for LLM Interactions

Huntr - The World's First AI/Machine Learning Bug Bounty Platform

50 episodios

#Tech #MLSecOps.com #Machine Learning #Artificial Intelligence #Cybersecurity #MLSecOps #Ai Security #InfoSec #Generative AI #Llm #ChatGPT #Responsible Ai #CISO

The MLSecOps Podcast

Evaluating Real-World Adversarial ML Attack Risks and Effective Management: Robustness vs Non-ML Mitigations

The MLSecOps Podcast

published hace un año

MP3•Episodio en casa

Send us a text

Recon: Automated Red Teaming for GenAI

Protect AI’s ML Security-Focused Open Source Tools

LLM Guard Open Source Security Toolkit for LLM Interactions

Huntr - The World's First AI/Machine Learning Bug Bounty Platform

50 episodios

#Tech #MLSecOps.com #Machine Learning #Artificial Intelligence #Cybersecurity #MLSecOps #Ai Security #InfoSec #Generative AI #Llm #ChatGPT #Responsible Ai #CISO

Tüm bölümler

The MLSecOps Podcast

1
What’s Hot in AI Security at RSA Conference 2025? 24:14

hace 2 días24:14

24:14

Send us a text What’s really hot at RSA Conference 2025? MLSecOps Community Manager Madi Vorbrich sits down with Protect AI Co‑Founder Daryan “D” Dehghanpisheh for a rapid rundown of must‑see sessions, booth events, and emerging AI‑security trends—from GenAI agents to zero‑trust AI and million‑model scans. Use this episode to build a bullet‑proof RSA agenda before you land in San Francisco. Full transcript with links to resources available at https://mlsecops.com/podcast/whats-hot-in-ai-security-at-rsa-conference-2025 Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Unpacking the Cloud Security Alliance AI Controls Matrix 35:53

hace 7 días35:53

35:53

Send us a text In this episode of the MLSecOps Podcast, we sit down with three expert contributors from the Cloud Security Alliance’s AI Controls Matrix working group. They reveal how this newly released framework addresses emerging AI threats—like model poisoning and adversarial manipulation—through robust technical controls, detailed implementation guidelines, and clear auditing strategies. Full transcript with links to resources available at https://mlsecops.com/podcast/unpacking-the-cloud-security-alliance-ai-controls-matrix Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
From Pickle Files to Polyglots: Hidden Risks in AI Supply Chains 41:21

hace 21 días41:21

41:21

Send us a text Join Keith Hoodlet from Trail of Bits as he dives into AI/ML security, discussing everything from prompt injection and fuzzing techniques to bias testing and compliance challenges. Full transcript with links to resources available at https://mlsecops.com/podcast/from-pickle-files-to-polyglots-hidden-risks-in-ai-supply-chains Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Rethinking AI Red Teaming: Lessons in Zero Trust and Model Protection 36:52

hace 5 weeks36:52

36:52

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/rethinking-ai-red-teaming-lessons-in-zero-trust-and-model-protection This episode is a follow up to Part 1 of our conversation with returning guest Brian Pendleton, as he challenges the way we think about red teaming and security for AI. Continuing from last week’s exploration of enterprise AI adoption and high-level security considerations, the conversation now shifts to how red teaming, zero trust, and privacy concerns intertwine with AI’s unique risks. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Security: Map It, Manage It, Master It 41:18

hace 6 weeks41:18

41:18

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/ai-security-map-it-manage-it-master-it In part one of our two-part MLSecOps Podcast episode, security veteran Brian Pendleton takes us from his early hacker days to the forefront of AI security. Brian explains why mapping every AI integration is essential for uncovering vulnerabilities. He also dives into the benefits of using SBOMs over model cards for risk management and stresses the need to bridge the gap between ML and security teams to protect your enterprise AI ecosystem. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Agentic AI: Tackling Data, Security, and Compliance Risks 23:22

hace 7 weeks23:22

23:22

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/agentic-ai-tackling-data-security-and-compliance-risks Join host Diana Kelley and CTO Dr. Gina Guillaume-Joseph as they explore how agentic AI, robust data practices, and zero trust principles drive secure, real-time video analytics at Camio. They discuss why clean data is essential, how continuous model validation can thwart adversarial threats, and the critical balance between autonomous AI and human oversight. Dive into the world of multimodal modeling, ethical safeguards, and what it takes to ensure AI remains both innovative and risk-aware. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Vulnerabilities: ML Supply Chains to LLM and Agent Exploits 24:08

hace 8 weeks24:08

24:08

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/ai-vulnerabilities-ml-supply-chains-to-llm-and-agent-exploits Join host Dan McInerney and AI security expert Sierra Haex as they explore the evolving challenges of AI security. They discuss vulnerabilities in ML supply chains, the risks in tools like Ray and untested AI model files, and how traditional security measures intersect with emerging AI threats. The conversation also covers the rise of open-source models like DeepSeek and the security implications of deploying autonomous AI agents, offering critical insights for anyone looking to secure distributed AI systems. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Implementing Enterprise AI Governance: Balancing Ethics, Innovation & Risk for Business Success 38:39

hace 10 weeks38:39

38:39

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/implementing-a-robust-ai-governance-framework-for-business-success In this episode of the MLSecOps podcast, host Charlie McCarthy sits down with Chris McClean, Global Lead for Digital Ethics at Avanade, to explore the world of responsible AI governance. They discuss how ethical principles, risk management, and robust security practices can be integrated throughout the AI lifecycle—from design and development to deployment and oversight. Learn practical strategies for building resilient AI frameworks, understanding regulatory impacts, and driving innovation safely. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Unpacking Generative AI Red Teaming and Practical Security Solutions 51:53

hace 11 weeks51:53

51:53

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/unpacking-generative-ai-red-teaming-and-practical-security-solutions In this episode, we explore LLM red teaming beyond simple “jailbreak” prompts with special guest Donato Capitella, from WithSecure Consulting. You’ll learn why vulnerabilities live in context—how LLMs interact with users, tools, and documents—and discover best practices for mitigating attacks like prompt injection. Our guest also previews an open-source tool for automating security tests on LLM applications. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Security: Vulnerability Detection and Hidden Model File Risks 38:19

hace 19 weeks38:19

38:19

Send us a text In this episode of the MLSecOps Podcast , the team dives into the transformative potential of Vulnhuntr: zero shot vulnerability discovery using LLMs . Madison Vorbrich hosts Dan McInerney and Marcello Salvati to discuss Vulnhuntr’s ability to autonomously identify vulnerabilities, including zero-days, using large language models (LLMs) like Claude. They explore the evolution of AI tools for security, the gap between traditional and AI-based static code analysis, and how Vulnhuntr enables both developers and security teams to proactively safeguard their projects. The conversation also highlights Protect AI’s bug bounty platform, huntr.com, and its expansion into model file vulnerabilities (MFVs), emphasizing the critical need to secure AI supply chains and systems. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Governance Essentials: Empowering Procurement Teams to Navigate AI Risk 37:41

hace 24 weeks37:41

37:41

Send us a text Full transcript with links to resources available at https://mlsecops.com/podcast/ai-governance-essentials-empowering-procurement-teams-to-navigate-ai-risk . In this episode of the MLSecOps Podcast, Charlie McCarthy from Protect AI sits down with Dr. Cari Miller to discuss the evolving landscapes of AI procurement and governance. Dr. Miller shares insights from her work with the AI Procurement Lab and ForHumanity, delving into the essential frameworks and strategies needed to mitigate risks in AI acquisitions. They cover the AI Procurement Risk Management Framework, practical ways to ensure transparency and accountability, and how the September 2024 OMB Memo M-24-18 is guiding AI acquisition in government. Dr. Miller also emphasizes the importance of cross-functional collaboration and AI literacy to support responsible AI procurement and deployment in organizations of all types. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Crossroads: AI, Cybersecurity, and How to Prepare for What's Next 33:15

hace 25 weeks33:15

33:15

Send us a text In this episode of the MLSecOps Podcast, Distinguished Engineer Nicole Nichols from Palo Alto Networks joins host and Machine Learning Scientist Mehrin Kiani to explore critical challenges in AI and cybersecurity. Nicole shares her unique journey from mechanical engineering to AI security, her thoughts on the importance of clear AI vocabularies, and the significance of bridging disciplines in securing complex systems. They dive into the nuanced definitions of AI fairness and safety, examine emerging threats like LLM backdoors, and discuss the rapidly evolving impact of autonomous AI agents on cybersecurity defense. Nicole’s insights offer a fresh perspective on the future of AI-driven security, teamwork, and the growth mindset essential for professionals in this field. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Beyond the Hype: Lessons from Cloud on Risk and Security 41:06

hace 29 weeks41:06

41:06

Send us a text On this episode of the MLSecOps Podcast, we’re bringing together two cybersecurity legends. Our guest is the inimitable Caleb Sima, who joins us to discuss security considerations for building and using AI, drawing on his 25+ years of cybersecurity experience. Caleb's impressive journey includes co-founding two security startups acquired by HP and Lookout, serving as Chief Security Officer at Robinhood, and currently leading cybersecurity venture studio WhiteRabbit & chairing the Cloud Security Alliance AI Safety Initiative . Hosting this episode is Diana Kelley (CISO, Protect AI ) an industry powerhouse with a long career dedicated to cybersecurity, and a longtime host on this show. Together, Caleb and Diana share a thoughtful discussion full of unique insights for the MLSecOps Community of learners. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Generative AI Prompt Hacking and Its Impact on AI Security & Safety 31:59

hace 31 weeks31:59

31:59

Send us a text Welcome to Season 3 of the MLSecOps Podcast, brought to you by Protect AI ! In this episode, MLSecOps Community Manager Charlie McCarthy speaks with Sander Schulhoff, co-founder and CEO of Learn Prompting. Sander discusses his background in AI research, focusing on the rise of prompt engineering and its critical role in generative AI. He also shares insights into prompt security, the creation of LearnPrompting.org, and its mission to democratize prompt engineering knowledge. This episode also explores the intricacies of prompting techniques, "prompt hacking," and the impact of competitions like HackAPrompt on improving AI safety and security. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
The MLSecOps Podcast Season 2 Finale 40:54

hace 33 weeks40:54

40:54

Send us a text This compilation contains highlights from episodes throughout Season 2 of the MLSecOps Podcast , and it's a great one for community members who are new to the show. If there is a clip from this highlights reel that is especially interesting to you, you can note the name of the original episode that the clip came from and easily go check out that full length episode for a deeper dive. Extending enormous thanks to everyone who has supported this show, including our audience, Protect AI hosts, and stellar expert guests. Stay tuned for Season 3! Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Exploring Generative AI Risk Assessment and Regulatory Compliance 37:37

hace 39 weeks37:37

37:37

Send us a text In this episode of the MLSecOps Podcast we have the honor of talking with David Rosenthal, Partner at VISCHER (Swiss Law, Tax & Compliance). David is also an author & former software developer, and lectures at ETH Zürich & the University of Basel. He has more than 25 years of experience in data & technology law and kindly joined the show to discuss a variety of AI regulation topics, including the EU Artificial Intelligence Act, generative AI risk assessment, and challenges related to organizational compliance with upcoming AI regulations. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
MLSecOps Culture: Considerations for AI Development and Security Teams 38:44

hace 42 weeks38:44

38:44

Send us a text In this episode, we had the pleasure of welcoming Co-Founder and CISO of Weights & Biases, Chris Van Pelt, to the MLSecOps Podcast. Chris discusses a range of topics with hosts Badar Ahmed and Diana Kelley, including the history of how W&B was formed, building a culture of security & knowledge sharing across teams in an organization, real-world ML and GenAI security concerns, data lineage and tracking, and upcoming features in the Weights & Biases platform for enhancing security. More about our guest speaker: Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Practical Offensive and Adversarial ML for Red Teams 35:24

hace 44 weeks35:24

35:24

Send us a text Next on the MLSecOps Podcast, we have the honor of highlighting one of our MLSecOps Community members and Dropbox™ Red Teamers, Adrian Wood. Adrian joined Protect AI threat researchers, Dan McInerney and Marcello Salvati, in the studio to share an array of insights, including what inspired him to create the Offensive ML (aka OffSec ML) Playbook, and diving into categories like adversarial machine learning (ML), offensive/defensive ML, and supply chain attacks. The group also discusses dual uses for "traditional" ML and LLMs in the realm of security, the rise of agentic LLMs, and the potential for crown jewel data leakage via model malware (i.e. highly valuable and sensitive data being leaked out of an organization due to malicious software embedded within machine learning models or AI systems). Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Expert Talk from RSA Conference: Securing Generative AI 25:42

hace 48 weeks25:42

25:42

Send us a text In this episode, host Neal Swaelens (EMEA Director of Business Development, Protect AI) catches up with Ken Huang, CISSP at RSAC 2024 to talk about security for generative AI. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Practical Foundations for Securing AI 38:10

hace 49 weeks38:10

38:10

Send us a text In this episode of the MLSecOps Podcast, we delve into the critical world of security for AI and machine learning with our guest Ron F. Del Rosario, Chief Security Architect and AI/ML Security Lead at SAP ISBN. The discussion highlights the contextual knowledge gap between ML practitioners and cybersecurity professionals, emphasizing the importance of cross-collaboration and foundational security practices. We explore the contrasts of security for AI to that for traditional software, along with the risk profiles of first-party vs. third-party ML models. Ron sheds light on the significance of understanding your AI system's provenance, having necessary controls, and audit trails for robust security. He also discusses the "Secure AI/ML Development Framework" initiative that he launched internally within his organization, featuring a lean security checklist to streamline processes. We hope you enjoy this thoughtful conversation! Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Evaluating RAG and the Future of LLM Security: Insights with LlamaIndex 31:04

hace 52 weeks31:04

31:04

Send us a text In this episode of the MLSecOps Podcast, host Neal Swaelens, along with co-host Oleksandr Yaremchuk, sit down with special guest Simon Suo, co-founder and CTO of LlamaIndex. Simon shares insights into the development of LlamaIndex, a leading data framework for orchestrating data in large language models (LLMs). Drawing from his background in the self-driving industry, Simon discusses the challenges and considerations of integrating LLMs into various applications, emphasizing the importance of contextualizing LLMs within specific environments. The conversation delves into the evolution of retrieval-augmented generation (RAG) techniques and the future trajectory of LLM-based applications. Simon comments on the significance of balancing performance with cost and latency in leveraging LLM capabilities, envisioning a continued focus on data orchestration and enrichment. Addressing LLM security concerns, Simon emphasizes the critical need for robust input and output evaluation to mitigate potential risks. He discusses the potential vulnerabilities associated with LLMs, including prompt injection attacks and data leakage, underscoring the importance of implementing strong access controls and data privacy measures. Simon also highlights the ongoing efforts within the LLM community to address security challenges and foster a culture of education and awareness. As the discussion progresses, Simon introduces LlamaCloud, an enterprise data platform designed to streamline data processing and storage for LLM applications. He emphasizes the platform's tight integration with the open-source LlamaIndex framework, offering users a seamless transition from experimentation to production-grade deployments. Listeners will also learn about LlamaIndex's parsing solution, LlamaParse. Join us to learn more about the ongoing journey of innovation in large language model-based applications, while remaining vigilant about LLM security considerations. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Threat Research: Spotlight on the Huntr Community 31:48

hace 1 year31:48

31:48

Send us a text Learn about the world’s first bug bounty platform for AI & machine learning, huntr , including how to get involved! This week’s featured guests are leaders from the huntr community (brought to you by Protect AI): Dan McInerney, Lead AI Threat Researcher Marcello Salvati, Sr. Engineer & Researcher Madison Vorbrich, Community Manager Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Securing AI: The Role of People, Processes & Tools in MLSecOps 37:16

hace 1 year37:16

37:16

Send us a text In this episode of The MLSecOps Podcast hosted by Daryan Dehghanpisheh (Protect AI) and special guest-host Martin Stanley, CISSP (Cybersecurity and Infrastructure Security Agency), we delve into critical aspects of AI security and operations. This episode features esteemed guests, Gary Givental (IBM) and Kaleb Walton (FICO). The group's discussion unfolds with insights into the evolving field of Machine Learning Security Operations, aka, MLSecOps. A recap of CISA's most recent Secure by Design and Secure AI initiatives sets the stage for the a dialogue that explores the parallels between MLSecOps and DevSecOps. The episode goes on to illuminate the challenges of securing AI systems, including data integrity and third-party dependencies. The conversation also travels to the socio-technical facets of AI security, explores MLSecOps and AI security posture roles within an organization, and the interplay between people, processes, and tools essential to successful MLSecOps implementation. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
ReDoS Vulnerability Reports: Security Relevance vs. Noisy Nuisance 35:30

hace 1 year35:30

35:30

Send us a text In this episode, we delve into a hot topic in the bug bounty world: ReDoS (Regular Expression Denial of Service) reports. Inspired by reports submitted by the huntr AI/ML bug bounty community and an insightful blog piece by open source expert, William Woodruff (Engineering Director, Trail of Bits), this conversation explores: Are any ReDoS vulnerabilities worth fixing? Triaging and the impact of ReDoS reports on software maintainers. The challenges of addressing ReDoS vulnerabilities amidst developer fatigue and resource constraints. Analyzing the evolving trends and incentives shaping the rise of ReDoS reports in bug bounty programs, and their implications for severity assessment. Can LLMs be used to help with code analysis? Tune in as we dissect the dynamics of ReDoS, offering insights into its implications for the bug hunting community and the evolving landscape of security for AI/ML. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Finding a Balance: LLMs, Innovation, and Security 41:56

hace 1 year41:56

41:56

Send us a text In this episode of The MLSecOps Podcast, special guest, Sandy Dunn , joins us to discuss the dynamic world of large language models (LLMs) and the equilibrium of innovation and security. Co-hosts, Daryan “D” Dehghanpisheh and Dan McInerney talk with Sandy about the nuanced challenges organizations face in managing LLMs while mitigating AI risks. Exploring the swift pace of innovation juxtaposed with the imperative of maintaining robust security measures, the trio examines the critical need for organizations to adapt their security posture management to include considerations for AI usage. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Secure AI Implementation and Governance 38:37

hace 1 year38:37

38:37

Send us a text In this episode of The MLSecOps Podcast, Nick James , CEO of WhitegloveAI dives in with show host, Chris King , Head of Product at Protect AI , to offer enlightening insights surrounding: - AI Governance - ISO - International Organization for Standardization ISO/IEC 42001:2023-Information Technology, Artificial Intelligence Management System - Continuous improvement for AI security Thanks for listening! Find more episodes and transcripts at https://bit.ly/MLSecOpsPodcast. Additional MLSecOps and AI Security tools and resources to check out: Protect AI Radar (https://bit.ly/ProtectAIRadar) ModelScan (https://bit.ly/ModelScan) Protect AI’s ML Security-Focused Open Source Tools (https://bit.ly/ProtectAIGitHub) Huntr - The World's First AI/Machine Learning Bug Bounty Platform (https://bit.ly/aimlhuntr) Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Risk Management and Enhanced Security Practices for AI Systems 38:08

hace 1 year38:08

38:08

Send us a text In this episode of The MLSecOps Podcast, VP Security and Field CISO of Databricks, Omar Khawaja, joins the CISO of Protect AI, Diana Kelley. Together, Diana and Omar discuss a new framework for understanding AI risks, fostering a security-minded culture around AI, building the MLSecOps dream team, and some of the challenges that Chief Information Security Officers (CISOs) and other business leaders face when assessing the risk to their AI/ML systems. Get the scoop on Databricks’ new AI Security Framework on The MLSecOps Podcast. To learn more about the framework, contact cybersecurity@databricks.com. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Evaluating Real-World Adversarial ML Attack Risks and Effective Management: Robustness vs Non-ML Mitigations 41:19

hace 1 year41:19

41:19

Send us a text In this episode, co-hosts Badar Ahmed and Daryan Dehghanpisheh are joined by Drew Farris (Principal, Booz Allen Hamilton) and Edward Raff (Chief Scientist, Booz Allen Hamilton) to discuss themes from their paper, " You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks ," co-authored with Michael Benaroch. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
From Risk to Responsibility: Violet Teaming in AI; With Guest: Alexander Titus 43:20

hace 1 year43:20

43:20

Send us a text In this episode, the founder and CEO of The In Vivo Group , Alexander Titus, joins show hosts Diana Kelley and Daryan Dehghanpisheh to discuss themes from his forward-thinking paper, " The Promise and Peril of Artificial Intelligence -- Violet Teaming Offers a Balanced Path Forward ," authored with Adam H. Russell. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Cybersecurity of Tomorrow: Exploring the Future of Security and Governance for AI Systems; With Guest: Martin Stanley, CISSP 39:45

hace 2 years39:45

39:45

Send us a text *This episode is also available in video format! Click to watch the full YouTube video. * Welcome to Season 2 of The MLSecOps Podcast! In this episode, we joined Strategic Technology Branch Chief, Martin Stanley, CISSP , from the Cybersecurity and Infrastructure Security Agency (CISA), to celebrate 20 years of Cybersecurity Awareness Month, as well as hear his expert and thoughtful insights about CISA initiatives, partnering with the National Institute of Standards and Technology (NIST) to promote the adoption of their AI Risk Management Framework, AI security and governance, and much more. We are so grateful to Martin for joining us for this enlightening talk! Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI/ML Security in Retrospect: Insights from Season 1 of The MLSecOps Podcast (Part 2) 42:28

hace 2 years42:28

42:28

Send us a text *This episode is also available in video format! Click to watch the full YouTube video. * Welcome back, everyone, to The MLSecOps Podcast. We’re thrilled to have you with us for Part 2 of our two-part season finale, as we wrap up Season 1 and look forward to an exciting and revamped Season 2. In this two-part season recap, we’ve been revisiting some of the most captivating discussions from our first season, offering an overview on essential topics related to AI and machine learning security. Part 1 of this series focused on topics like adversarial machine learning, ML supply chain vulnerabilities, and red teaming for AI/ML. Here in Part 2, we've handpicked some standout moments from Season 1 episodes that will take you on a tour through categories such as model provenance; governance, risk, & compliance; and Trusted AI. Our wonderful guests on the show delve into topics like defining responsible AI, bias detection and prevention, model fairness, AI audits, incident response plans, privacy engineering, and the significance of data in MLSecOps . These episodes have been a testament to the expertise and insights of our fantastic guests, and we're excited to share their wisdom with you once again. Whether you're a long-time listener or joining us for the first time, there's something here for everyone, and if you missed the full-length versions of any of these thought-provoking discussions or simply want to revisit them, you can find links to the full episodes and transcripts on our website at www.mlsecops.com/podcast . Chapters: 0:00 Opening 0:25 Intro by Charlie McCarthy 2:29 S1E9 with Guest Diya Wynn 6:32 S1E4 with Guest Dr. Cari Miller, CMP, FHCA 11:03 S1E17 with Guest Nick Schmidt 16:46 S1E7 with Guest Shea Brown, PhD 22:06 S1E8 with Guest Patrick Hall 26:12 S1E14 with Guest Katharine Jarmul 32:01 S1E13 with Guest Jennifer Prendki, PhD 36:44 S1E18 with Guest Rob van der Veer Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI/ML Security in Retrospect: Insights from Season 1 of The MLSecOps Podcast (Part 1) 37:10

hace 2 years37:10

37:10

Send us a text *This episode is also available in video format! Click to watch the full YouTube video. * Welcome to the final episode of the first season of The MLSecOps Podcast, brought to you by the team at Protect AI . In this two-part episode, we’ll be taking a look back at some favorite highlights from the season where we dove deep into machine learning security operations. In this first part, we’ll be revisiting clips related to things like adversarial machine learning; how malicious actors can use AI to fool machine learning systems into making incorrect decisions; supply chain vulnerabilities; and red teaming for AI/ML, including how security professionals might simulate attacks on their own systems to detect and mitigate vulnerabilities. If you’re new to the show, or if you could use a refresher on any of these topics, this episode is for you, as it’s a great place for listeners to start their learning journey with us and work backwards based on individual interests. And when something in this recap piques your interest, be sure to check out the transcript for links to the full-length episodes where each of these clips came from. You can visit the website and read the transcripts at www.mlsecops.com/podcast. So now, we invite you to sit back, relax, and enjoy this Season 1 recap of some of the most important MLSecOps topics of the year. And stay tuned for part 2 of this episode, where we’ll be revisiting MLSecOps conversations surrounding governance, risk, and compliance, model provenance, and Trusted AI. Thanks for listening. Chapters: 0:00 Opening 0:25 Intro by Charlie McCarthy, MLSecOps Community Leader 2:15 S1E1 with Guest Disesdi Susanna Cox 5:08 S1E2 with Guest Dr. Florian Tramèr 10:16 S1E3 with Guest Pin-Yu Chen, PhD 13:18 S1E5 with Guest Christina Liaghati, PhD 17:59 S1E6 with Guest Johann Rehberger 22:10 S1E10 with Guest Kai Greshake 27:14 S1E11 with Guest Shreya Rajpal 31:45 S1E12 with Guest Apostol Vassilev 36:36 End/Credits Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
A Holistic Approach to Understanding the AI Lifecycle and Securing ML Systems: Protecting AI Through People, Processes & Technology; With Guest: Rob van der Veer 29:25

hace 2 years29:25

29:25

Send us a text Joining us for the first time as a guest host is Protect AI ’s CEO and founder, Ian Swanson . Ian is joined this week by Rob van der Veer , a pioneer in AI and security. Rob gave a presentation at Global AppSec Dublin earlier this year called “Attacking and Protecting Artificial Intelligence” which was a large inspiration for this episode. In it, Rob talks about the lack of security considerations and processes in AI production systems compared to traditional software development, and the unique challenges and particularities of building security into AI and machine learning systems. Together in this episode, Ian and Rob dive into things like practical threats to ML systems, the transition from MLOps to MLSecOps, the [upcoming] ISO 5338 standard on AI engineering, and what organizations can do if they are looking to mature their AI/ML security practices. This is a great dialogue and exchange of ideas overall between two super knowledgeable people in this industry. So thank you so much to Ian and to Rob for joining us on The MLSecOps Podcast this week. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
ML Model Fairness: Measuring and Mitigating Algorithmic Disparities; With Guest: Nick Schmidt 35:33

hace 2 years35:33

35:33

Send us a text This week we’re talking about the role of fairness in AI/ML. It is becoming increasingly apparent that incorporating fairness into our AI systems and machine learning models while mitigating bias and potential harms is a critical challenge. Not only that, it’s a challenge that demands a collective effort to ensure the responsible, secure, and equitable development of AI and machine learning systems. But what does this actually mean in practice? To find out, we spoke with Nick Schmidt , the Chief Technology and Innovation Officer at SolasAI . In this week’s episode, Nick reviews some key principles related to model governance and fairness, from things like accountability and ownership all the way to model deployment and monitoring. He also discusses real life examples of when machine learning algorithms have demonstrated bias and disparity, along with how those outcomes could be harmful to individuals or groups. Later in the episode, Nick offers some insightful advice for organizations who are assessing their AI security risk related to algorithmic disparities and unfair models. Additional tools and resources to check out: AI Radar ModelScan NB Defense Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Exploring AI/ML Security Risks: At Black Hat USA 2023 with Protect AI 35:20

hace 2 years35:20

35:20

Send us a text Watch the video for this episode at: https://mlsecops.com/podcast/exploring-ai/ml-security-risks-at-black-hat-usa-2023 This episode of The MLSecOps Podcast features expert security leaders who sat down at Black Hat USA 2023 last week with team members from Protect AI to talk about various facets of AI and machine learning security: - What is the overall state of the AI/ML security realm at this time? - What is currently the largest threat to AI and machine learning security? - What is the most important thing we need to focus on at this time in the machine learning security space? - Plus much more! Guest Speaker Information: Adam Nygate Adam is the founder of huntr .dev (recently acquired by Protect AI ), the place to protect open-source software, where they not only provide bug and fix bounties, but have built a platform that automates triaging, creating a noise-free environment for maintainers. Adam’s work with the huntr.dev community of over 10k security contributors enabled them to become the 5th largest CNA in the world in 2022. Adam was included in the OpenUK’s 2022 Honours List and has been featured in articles by AWS, Nominet and on podcasts such as Zero Hour and Absolute AppSec. Daniel Miessler "Daniel is the founder of Unsupervised Learning, a company focused on building products that help companies, organizations, and people identify, articulate, and execute on their purpose in the world. Daniel has over 20 years of experience in Cybersecurity, and has spent the last several years focused on applying AI to business and human problems." Adam Shostack Check out Adam's fantastic book, "Threats: What Every Engineer Should Learn From Star Wars!" Adam is a leading expert on threat modeling, a consultant, expert witness, and game designer. Christina Liaghati, PhD Dr. Liaghati is the AI Strategy Execution & Operations Manager, AI & Autonomy Innovation Center at MITRE , and a wealth of knowledge about all things MITRE ATLAS and the ML system attack chain. She is a highly sought-after speaker and is a repeat guest of ours on The MLSecOps Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Everything You Need to Know About Hacker Summer Camp 2023 38:59

hace 2 years38:59

38:59

Send us a text Welcome back to The MLSecOps Podcast for this week's episode, “Everything You Need to Know About Hacker Summer Camp 2023.” This week, our show is hosted by Protect AI's Chief Information Security Officer, Diana Kelley, and Diana talks with two more longtime security experts, Chloé Messdaghi and Dan McInerney, about all things related to what the security research community fondly refers to as Hacker Summer Camp. The group discusses various events held throughout the course of this exciting week in Las Vegas, including what to expect at Black Hat [USA 2023] and DEF CON [31] . (1:21) What is Hacker Summer Camp Week and what are the various events and Cons that take place during that time? (3:58) It’s my first time attending Black Hat, DEF CON, Hacker Summer Camp Week, etc.: where can I find groups to attend with or to help me navigate where to go and what events to attend? (9:53) Advice: if it’s my first time attending Black Hat, what other advice is there for me? What should I be thinking about? (13:25) If I attend Black Hat, does that mean I’m automatically able to attend DEF CON or how does that work? (TL;DR separate passes are needed for each event) (14:14) Are certain personas more welcomed at specific conferences? (15:49) What are some interesting panel talks we should know about? (20:53) There are a couple of other conferences going on during “Summer Camp Week” - BSides Las Vegas, Squadcon, Diana Initiative. What are those? When are they taking place? Can I go to all of them? How does that work? (23:26) What AI/ML security trends are happening? What should I be looking for at Black Hat and DEF CON this year in terms of talks and research? (28:55) How can I determine if a particular talk is going to be worth my time? (32:54) Any other tips on how to stay healthy and safe (both physical and electronic safety) throughout the week? Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Privacy Engineering: Safeguarding AI & ML Systems in a Data-Driven Era; With Guest Katharine Jarmul 46:44

hace 2 years46:44

46:44

Send us a text Welcome to The MLSecOps Podcast, where we dive deep into the world of machine learning security operations. In this episode, we talk with the renowned Katharine Jarmul. Katharine is a Principal Data Scientist at Thoughtworks , and the author of the popular new book, Practical Data Privacy . Katharine also writes a blog titled, Probably Private , where she writes about data privacy, data security, and the intersection of data science and machine learning. We cover a lot of ground in this conversation; from the more general data privacy and security risks associated with ML models, to more specific cases such as the case with OpenAI’s ChatGPT. We also touch on things like how GDPR and other regulatory frameworks put a spotlight on the privacy concerns we all have when it comes to the massive amount of data collected by models. Where does the data come from? How is it collected? Who gives consent? What if somebody wants to have their data removed? We also get into how organizations and professionals such as business leaders, data scientists, and ML practitioners can address these challenges when it comes to risks surrounding data, privacy, security, and reputation. We also explore the practices and processes that need to be implemented in order to integrate “ Privacy by Design ” into the machine learning lifecycle. Katharine is a wealth of knowledge and insight into these data privacy issues. As always, thanks for listening to the podcast, for reading the transcript, and supporting the show in any way you can. With that, we hope you enjoy our conversation with Katharine Jarmul. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
The Intersection of MLSecOps and DataPrepOps; With Guest: Jennifer Prendki, PhD 34:40

hace 2 years34:40

34:40

Send us a text On this week’s episode from The MLSecOps Podcast, we have the pleasure of hearing from Dr. Jennifer Prendki , founder and CEO of Alectio - The DataPrepOps Company. Alectio’s name comes from a blend of the acronym “AL,” standing for Active Learning, and the Latin term for the word “selection,” which is “lectio.” In this episode, Dr. Prendki defines DataPrepOps for us and describes its contrasts to MLOps, along with how DataPrepOps intersects with MLSecOps best practices. She also discusses data quality, security risks in data science, and the role that data curation plays in helping to mitigate security risks in ML models. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
The Evolved Adversarial ML Landscape; With Guest: Apostol Vassilev, NIST 30:30

hace 2 years30:30

30:30

Send us a text In this episode, we explore the National Institute of Standards and Technology (NIST) white paper, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations . The report is co-authored by our guest for this conversation; Apostol Vassilev , NIST Research Team Supervisor. Apostol provides insights into the motivations behind this initiative and the collaborative research methodology employed by the NIST team. Apostol shares with us that this taxonomy and terminology report is part of the Trustworthy & Responsible AI Resource Center that NIST is developing. Additional tools in the resource center include NIST’s AI Risk Management Framework (RMF) , the OECD-NIST Catalogue of AI Tools and Metrics , and another crucial publication that Apostol co-authored called Towards a Standard for Identifying and Managing Bias in Artificial Intelligence . The conversation then focuses on the evolution of adversarial ML (AdvML) attacks, including prominent techniques like prompt injection attacks, as well as other emerging threats amidst the rise of large language model applications. Apostol discusses the changing AI and computing infrastructure and the scale of defenses required as a result of these changes. Concluding the episode, Apostol shares thoughts on enhancing ML security practices and invites stakeholders to contribute to the ongoing development of the AdvML taxonomy and terminology white paper. Join us now for a thought-provoking discussion that sheds light on NIST's efforts to further define the terminology of adversarial ML and develop a comprehensive taxonomy of concepts that will aid industry leaders in creating additional standards and guides. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Navigating the Challenges of LLMs: Guardrails AI to the Rescue; With Guest: Shreya Rajpal 39:16

hace 2 years39:16

39:16

Send us a text In “Navigating the Challenges of LLMs: Guardrails to the Rescue,” Protect AI Co-Founders, Daryan Dehghanpisheh and Badar Ahmed , interview the creator of Guardrails AI, Shreya Rajpal . Guardrails AI is an open source package that allows users to add structure, type, and quality guarantees to the outputs of large language models (LLMs). In this highly technical discussion, the group digs into Shreya’s inspiration for starting the Guardrails project, the challenges of building a deterministic “guardrail” system on top of probabilistic large language models, and the challenges in general (both technical and otherwise) that developers face when building applications for LLMs. If you’re an engineer or developer in this space looking to integrate large language models into the applications you’re building, this episode is a must-listen and highlights important security considerations. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Indirect Prompt Injections and Threat Modeling of LLM Applications; With Guest: Kai Greshake 36:14

hace 2 years36:14

36:14

Send us a text This talk makes it increasingly clear. The time for machine learning security operations - MLSecOps - is now. In “Indirect Prompt Injections and Threat Modeling of LLM Applications,” (transcript here -> https://bit.ly/45DYMAG ) we dive deep into the world of large language model (LLM) attacks and security. Our conversation with esteemed cyber security engineer and researcher, Kai Greshake , centers around the concept of indirect prompt injections, a novel adversarial attack and vulnerability in LLM-integrated applications, which Kai has explored extensively. Our host, Daryan Dehghanpisheh , is joined by special guest-host (Red Team Director and prior show guest) Johann Rehberger to discuss Kai’s research, including the potential real-world implications of these security breaches. They also examine contrasts to traditional security injection vulnerabilities like SQL injections. The group also discusses the role of LLM applications in everyday workflows and the increased security risks posed by their integration into various industry systems, including military applications. The discussion then shifts to potential mitigation strategies and the future of AI red teaming and ML security. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
Responsible AI: Defining, Implementing, and Navigating the Future; With Guest: Diya Wynn 33:17

hace 2 years33:17

33:17

Send us a text In this episode of The MLSecOps Podcast, Diya Wynn, Sr. Practice Manager in Responsible AI in the Machine Learning Solutions Lab at Amazon Web Services shares her background and the motivations that led her to pursue a career in Responsible AI. Diya shares her passion for work related to diversity, equity, and inclusion (DEI), and how Responsible AI offers a unique opportunity to merge her passion for DEI with what her core focus has always been: technology. She explores the definition of Responsible AI as an operating approach focused on minimizing unintended impact and maximizing benefits. The group also spends some time in this episode discussing Generative AI and its potential to perpetuate biases and raise ethical concerns. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
ML Security: AI Incident Response Plans and Enterprise Risk Culture; With Guest: Patrick Hall 38:49

hace 2 years38:49

38:49

Send us a text In this episode of The MLSecOps Podcast, Patrick Hall , co-founder of BNH.AI and author of " Machine Learning for High-Risk Applications, " discusses the importance of “responsible AI” implementation and risk management. He also shares real-world examples of incidents resulting from the lack of proper AI and machine learning risk management; supporting the need for governance, security, and auditability from an MLSecOps perspective. This episode also touches on the culture items and capabilities organizations need to build to have a more responsible AI implementation, the key technical components of AI risk management, and the challenges enterprises face when trying to implement responsible AI practices - including improvements to data science culture that some might suggest lacks authentic “science” and scientific practices. Also discussed are the unique challenges posed by large language models in terms of data privacy, bias management, and other incidents. Finally, Hall offers practical advice on using the NIST AI Risk Management Framework to improve an organization's AI security posture, and how BNH.AI can help those in risk management, compliance, general counsel and various other positions. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
AI Audits: Uncovering Risks in ML Systems; With Guest: Shea Brown, PhD 41:02

hace 2 years41:02

41:02

Send us a text Shea Brown, PhD explores with us the “W’s” and security practices related to AI and algorithm audits. What is included in an AI audit? Who is requesting AI audits and, conversely, who isn’t requesting them but should be? When should organizations request a third party audit of their AI/ML systems and machine learning algorithms? Why should they do so? What are some organizational risks and potential public harms that could result from not auditing AI/ML systems? What are some next steps to take if the results of your audit are unsatisfactory or noncompliant? Shea Brown, PhD ; is the Founder and CEO of BABL AI , and a faculty member in the Department of Physics & Astronomy at the University of Iowa. Thanks for checking out the MLSecOps Podcast! Get involved with the MLSecOps Community and find more resources at https://community.mlsecops.com . Additional tools and resources to check out: Protect AI Guardian: Zero Trust for ML Models Recon: Automated Red Teaming for GenAI Protect AI’s ML Security-Focused Open Source Tools LLM Guard Open Source Security Toolkit for LLM Interactions Huntr - The World's First AI/Machine Learning Bug Bounty Platform…

The MLSecOps Podcast

1
MLSecOps: Red Teaming, Threat Modeling, and Attack Methods of AI Apps; With Guest: Johann Rehberger 40:29

hace 2 years40:29