Dawn Song: Adversarial Machine Learning and Computer Security | Lex Fridman Podcast #95

Dawn Song is a professor of computer science at UC Berkeley with research interests in security, most recently with a focus on the intersection between computer security and machine learning.

the following is a conversation with Dan song a professor of computer science at UC Berkeley with research interests and computer security most recently with a focus on the intersection between security and machine learning this conversation was recorded before the outbreak of the pandemic for everyone feeling the medical psychological and financial burden of this crisis I’m sending love your way stay strong we’re in this together we’ll beat this thing this is the artificial intelligence podcast if you enjoy it subscribe on YouTube review it with five stars on Apple podcast supported on patreon or simply connect with me on Twitter Alex Friedman spelled the Fri D M a. m. as usual I’ll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation I hope that works for you it doesn’t hurt the listening experience this show is presented by cash app the number one finance app in the App Store when you get it use collects podcast cash app lets you send money to friends buy Bitcoin and invest in the stock market with as little as $1 since cash app does fractional share trading let me mention that the order execution algorithm that works behind the scenes to create the abstraction of fractional orders is an algorithmic marvel so big props the cash app engineers for solving a hard problem that in the end provides an easy interface that takes a step up to the next layer of abstraction over the stock market making trading more accessible for new investors and diversification much easier so again if you get cash app from the App Store Google Play and use the code lex podcast you get ten dollars in cash wrap will also donate ten dollars the first an organization that is helping to advanced robotics and STEM education for young people around the world and now here’s my conversation with dawn song systems will always have security vulnerabilities I started abroad almost philosophical level that’s a very good question I mean in general right it’s very difficult to write completely bug-free code and code that has no one in policy and also especially given that’s the definition for nobility is actually really proud it’s any type of attacks essentially an ax code can you know that’s can you can cut out the cost by vulnerabilities and the nature of attacks is always changing as well like new parts are coming up okay so for example in the past we talked about memory safety type of vulnerabilities we’re essentially tackers can exploit and the software and the take over control for how the code runs and then can launch attacks that way by accessing some aspect of the memory and be able to then alter the state of the program excite so for example in the example for buffer overflow then the attacker essentially actually causes essentially unintended changes in the states of the after program and then for example can then take over control flow after program and that the program to execute code that’s actually the the programming design intent so the attack can be a remote attack so they the attacker for example can can send in a malicious input to the program that just causes a program to completely then be compromised and then end up doing something that’s under the program and the attackers control and intention but that’s just one form of attacks and there are other forms of attacks like for example there are these side channels where attackers can try to learn from even just observing the outputs from the behaviors of the program try to infer certain secrets of the program so they essentially write the form of attacks it’s very very it’s very broad spectrum and in general from the security perspective we want to essentially provide as much guarantee as possible about the program’s security properties and so on so for example we talked about the provable guarantees of the program so for example there are ways we can use program analysis and form verification techniques to prove that a piece of code has no memory safety vulnerabilities what does that look like what does that proof is that just a dream for that’s applicable to small case examples is that possible to do two for real-world systems so actually I mean today I actually call it so we are entering the area of formally verified systems so in the community we have been working for the past decades in developing techniques and tools to do this type of program verification and and we have dedicated teams that have dedicated you know they’re like years sometimes even decades of their work in the space so as a result so we actually have a number of formally verify systems ranging from micro kernels to compilers to file systems to certain crypto you know libraries and so on and so it’s actually really wide ranging and it’s really exciting to see that people are recognizing the importance of having these formally verified systems with verified security so that’s great advancement that we see but on the other hand I think we do need to take all these in essentially with with the culture as well in the sense that’s just like I said the the type of vulnerability is very varied so we can form a very fine a software system to have certain set of security properties but they can still be vulnerable to other types of attacks and hence it’s that we continue to make progress in the in the space so just a quick to linger on the formal verification is that something you can do by looking at the code alone or is it something you have to run the code to to prove something so empirical verification can you look at the code just the code so that’s a very very question so in general for most program verification techniques is essentially try to verify the properties of the program statically and there are reasons for that too we can run the code to see for example using like in suffer testing with fasting techniques and also in certain even model checking techniques you can actually run the code but in general that only allows you to essentially verify or analyze the behaviors after program in certain and the certain situations and so most of the program verification techniques actually works statically what astatically mean that’s the running the code without writing the code yep so what sort of to return this is the big question if we can stand that for a little bit longer do you think there will always be security vulnerabilities you know that’s such a huge worry for people in the broad cyber security threat in the world it seems like the the tension between nations between groups the the Wars of the future might be fought in cyber security security that people worry about and so of course the nervousness is is this something that we can get a hold of in the future for our software systems so there’s a very funny quotes seeing security is job security we strive to make progress in building more secure systems and also making it easier and easier to build secure systems but given and the diversity the the various nature of attacks and also the interesting thing about security is that unlike in most other views essentially we are trying to hash applets improve a statement true but in this case yes trying to say that there is no attacks so even just this demon itself it’s not very well defined again given you know how vary the nature of the attacks can be it has there’s a challenge of security and also then naturally essentially it’s almost impossible to say that something a real-world system is a hundred percent no security vulnerabilities is there a particular and we’ll talk about different kinds of vulnerabilities it’s exciting ones very fascinating ones in the space of machine learning but is there a particular