Threat Models

Loading...

En provenance du cours de University of Minnesota

Nearest Neighbor Collaborative Filtering

197 notes

University of Minnesota

Nearest Neighbor Collaborative Filtering

197 notes

Cours 2 sur 5 dans Specialization Recommender Systems

In this course, you will learn the fundamental techniques for making personalized recommendations through nearest-neighbor techniques. First you will learn user-user collaborative filtering, an algorithm that identifies other people with similar tastes to a target user and combines their ratings to make recommendations for that user. You will explore and implement variations of the user-user algorithm, and will explore the benefits and drawbacks of the general approach. Then you will learn the widely-practiced item-item collaborative filtering algorithm, which identifies global product associations from user ratings, but uses these product associations to provide personalized recommendations based on a user's own product ratings.

À partir de la leçon

Advanced Collaborative Filtering Topics

The Cold Start Problem13:26

Recommending for Groups: Interview with Anthony Jameson14:20

Threat Models11:46

Explanations16:22

Explanations, Part II: Interview with Nava Tintarev17:15

Rencontrer les enseignants

Joseph A Konstan
Distinguished McKnight Professor and Distinguished University Teaching Professor
Computer Science and Engineering
Michael D. Ekstrand
Assistant Professor
Dept. of Computer Science, Boise State University

In this lecture, we're going to be talking about threat models.

We've talked some throughout the course about privacy, noise,

robustness for against attackers and the recommender.

This video is going to talk about how to think about the threat models,

the things that could damage your recommender system in a general way.

It'll deal with privacy, it'll deal with robustness.

We'll primarily be thinking about malicious behavior,

people trying to extract information they shouldn't have.

Or people trying to disrupt the system for some reason,

but it also has implications for benign problems such as inconstinent ratings from

users just due to the fact that users don't rate things consistently.

The core question that often comes up is what does it mean for

a recommender to be secure?

You want to talk about a recommender secure it's robust or

maybe it protects privacy.

But we need to think about these things carefully in order to better understand

exactly what kind of security or what kind of robustness or what kind of privacy

protection the recommender application or algorithm provides.

So we want to think about a threat model, where we're trying to protect something

that's important to the recommender or to users, from someone

who is trying to accomplish certain things, and has certain capabilities.

And, if we don't think about what capabilities we assumed that the attackers

has, and we should generally assume a generous set of capabilities.

Then it’s difficult to think about where our implementation might not be adequate.

And also if we don't think about what is it that we're trying to protect,

then it's difficult to formulate clearly exactly what protections we're providing.

And there may be things where we have multiple things we want to protect, but

protecting one makes another vulnerable.

So for most of the rest of this lecture, I'll go through several examples

of protections that a recommender might want to provide, and

talk about the threat model underlying those protections.

So we had Paul Reznick here, we talked about the influence limiter.

To protect recommender accuracy and neutrality, it's ability to

accurately recommend in an unbiased fashion items that users will like,

from malicious users who want to push, promote or kill particular

products that they might be associated with or associated with competitors too.

And what they can do is they can create fake account.

And they can populate these accounts with ratings and use these ratings to try to

promote or harm the particular product they're targeting.

And that's threat model, we want to protect accuracy neutrality from malicious

users who want to push our kill products and can create fake accounts.

And the influence limiter may provide protections from other threats as well,

but this is one of the key threat models in any kind of

protection from a shilling attack.

And so what it does is it requires users to prove themselves.

Malicious users have some threshold to cross before their ratings can be

incorporated into prediction, and that threshold is designed

to measure how much value they're providing in the recommendation process.

And so it effectively tries to make the system resilient to those users.

Alternative approaches are attempting to detect attacking users,

shilling users, and remove them from the rating data set or discount them.

There are other approaches to resilience as well,

such as making algorithms that don't have as direct a mapping,

from what the user does to the final recommendations.

But still do an effective job of accurate recommendation.

A more general, not necessarily more general, another aspect that can also

incorporate some more benign things is we want to protect the recommender

accuracy from users who either want to disrupt its quality.

Maybe they're just out to give the recommender a bad recommendation,

provide bad ratings so it gives users bad recommendations.

But this threat model, it turns out, also deals with users who

are just normal users, and by the fact of being people, and people aren't terribly

consistent in the way they express their preferences, they give noisy ratings.

And they can create profiles and ratings.

And this is basically a normal de-noising problem, dealing with either malicious or

natural noise.

Malicious noise is bad ratings introduced for

the purpose of disrupting a system or promoting or killing particular products.

Or a natural noise is just what naturally occurs in the fact

that we're measuring people's preference.

They both fit in this framing.

Recommender systems literatures, variety of techniques for

dealing with this detecting bad ratings.

Either just attempting to attack noisy ratings within a user's data set or

having users rate things again, and

using that to detect ratings that are particularly noisy.

We might also want to protect user privacy.

So maybe protect user data from other users of the system.

So we don't want one user to be able to figure out what another user's

ratings are.

There are some systems were user ratings are public.

For example,

Goodreads, your ratings are generally shared at least with your friends.

But other system ratings are not as public or they're publicly aggregated.

They are who provided what ratings is not public information.

So if we want to protect user ratings, information about that from

other users of the system who want to know the private opinions

of their fellow users, and they can create profiles and they can manipulate ratings.

One early recommend the privacy attack took advantage of the problem with

Pearson correlation and user-user collaborative filtering, where

the correlation between two users who've just rated one thing in common is one.

And with given a little bit of information about a user,

you can create a profile that targets them.

Creates a similarity of one, and can use that to extract their preference for

some item you want to know their preference of.

One mitigation strategy for this is to use a less transparent algorithm with

something like a matrix factorization, a singular value decomposition.

It's a lot harder to craft that target and attack profile, and

accurately obtain the information of a particular user in a system.

There's also considerations for user system privacy.

A lot of recommenders don't really consider this.

Goodreads knows all your ratings and the general user contract with Gooodreads,

that's fine.

But there are situations, there's been research work on recommendation,

where you protect information about the user,

including their preferences from the service provider itself.

Who wants to know users' characteristics, and

can analyze all the data that users give it.

And solving this problem is hard, and does require some more refinement and

exactly what we're trying to protect from who.

Are we trying to create a system where the system can do recommendations

without knowing the users' ratings or a particular user's ratings?

Are we trying to create a system where they know the users' ratings,

but can't in such a way that we try to protect

it from being able to infer othe datar about the user?

Or maybe we have a merchant and

we don't want them to be able to obtain too much user data.

But it's okay if the recommender service might know that data so

long as the merchant doesn't.

And there's a variety of ideas for providing this kind of privacy that some

of them work fairly well, some of them break down fairly quickly.

One, you could separate the recommender from the vendor.

Also, these ideas can be combined together, so

you can separate the recommender from the vendor.

So one organization sells things, another organization provides recommendations.

The recommender company doesn't get to find out what users bought,

the vendor doesn't get to find out the data that's being used in recommendations.

There's been a proposals to use trusted computing infrastructure to attest

recommender integrity.

So that the recommendation servers running on computers

that use cryptography and certifications from trusted third parties,

that they are implementing recommendation in a way

that preserves certain guarantees to the users of how their data is handled.

There's been proposals to pool ratings between users so different users

will pool their ratings together, present them then to the system and

get recommendations back, to hide themselves among fellow users.

Adding noise to ratings and profiles.

With some of this issues there can be fairly easy to go and

just de-noise the ratings.

So, if you're going to try that kind of strategy, you have to think about how

hard is going to be to just take out the noise that's being added.

Also, there's been work on decentralizing recommendations.

So that rather than having a centralize recommender server,

users collaborate with each other directly in order to compute recommendations.

And then also using things like homomorphic encryption which is

encryption protocols where you can do computation on the encrypted data, and

it is computation on the underlying real data.

So that you can provide an encrypted version of your profile to your friend or

to the service, that's encrypted under homomorphic encryption,

you do computation over that, you get the recommendations back and decrypt them.

That gets very, very expensive and

is often not practical to actually use in a deployed system,

but there's been some work starting to explore using it for recommendations.

So there's a variety of ideas to provide this kind of user-system privacy,

there's not a whole lot that's been widely deployed at present.

And in a lot of systems, the user may very well not care.

They might be fine with the system having access to their data

especially if they have a relationship of trust with the service provider.

So it's important to think carefully about the threats that you want to protect

from in your recommender system, and what threats your users might consider.

And if you're going to make privacy claims or robustness claims, or

security claims about your recommender algorithm,

to be careful in thinking about what protections you're actually providing.

Carefully think out the thread model to make sure that the claims

you're providing about the kind of privacy you preserve,

in both a research setting and in an industrial setting, and

the advertising claims that you make to your users about what your algorithm does.

Think carefully about the threat model to make sure the your claims are accurate.

And also,

it helps you to understand where the vulnerabilities in your system might be.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2019 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play