Silk Road forums

Discussion => Security => Topic started by: garry63 on August 15, 2013, 05:33 pm

Title: Encryption is less secure than we thought
Post by: garry63 on August 15, 2013, 05:33 pm
Quote
Encryption is less secure than we thought
For 65 years, most information-theoretic analyses of cryptographic systems have made a mathematical assumption that turns out to be wrong.
Larry Hardesty, MIT News Office

Information theory — the discipline that gave us digital communication and data compression — also put cryptography on a secure mathematical foundation. Since 1948, when the paper that created information theory first appeared, most information-theoretic analyses of secure schemes have depended on a common assumption.

Unfortunately, as a group of researchers at MIT and the National University of Ireland (NUI) at Maynooth, demonstrated in a paper presented at the recent International Symposium on Information Theory (view PDF), that assumption is false. In a follow-up paper being presented this fall at the Asilomar Conference on Signals and Systems, the same team shows that, as a consequence, the wireless card readers used in many keyless-entry systems may not be as secure as previously thought.

In information theory, the concept of information is intimately entwined with that of entropy. Two digital files might contain the same amount of information, but if one is shorter, it has more entropy. If a compression algorithm — such as WinZip or gzip — worked perfectly, the compressed file would have the maximum possible entropy. That means that it would have the same number of 0s and 1s, and the way in which they were distributed would be totally unpredictable. In information-theoretic parlance, it would be perfectly uniform.

Traditionally, information-theoretic analyses of secure schemes have assumed that the source files are perfectly uniform. In practice, they rarely are, but they’re close enough that it appeared that the standard mathematical analyses still held.

“We thought we’d establish that the basic premise that everyone was using was fair and reasonable,” says Ken Duffy, one of the researchers at NUI. “And it turns out that it’s not.” On both papers, Duffy is joined by his student Mark Christiansen; Muriel Médard, a professor of electrical engineering at MIT; and her student Flávio du Pin Calmon.

The problem, Médard explains, is that information-theoretic analyses of secure systems have generally used the wrong notion of entropy. They relied on so-called Shannon entropy, named after the founder of information theory, Claude Shannon, who taught at MIT from 1956 to 1978.

Shannon entropy is based on the average probability that a given string of bits will occur in a particular type of digital file. In a general-purpose communications system, that’s the right type of entropy to use, because the characteristics of the data traffic will quickly converge to the statistical averages. Although Shannon’s seminal 1948 paper dealt with cryptography, it was primarily concerned with communication, and it used the same measure of entropy in both discussions.

But in cryptography, the real concern isn’t with the average case but with the worst case. A codebreaker needs only one reliable correlation between the encrypted and unencrypted versions of a file in order to begin to deduce further correlations. In the years since Shannon’s paper, information theorists have developed other notions of entropy, some of which give greater weight to improbable outcomes. Those, it turns out, offer a more accurate picture of the problem of codebreaking.

When Médard, Duffy and their students used these alternate measures of entropy, they found that slight deviations from perfect uniformity in source files, which seemed trivial in the light of Shannon entropy, suddenly loomed much larger. The upshot is that a computer turned loose to simply guess correlations between the encrypted and unencrypted versions of a file would make headway much faster than previously expected.

“It’s still exponentially hard, but it’s exponentially easier than we thought,” Duffy says. One implication is that an attacker who simply relied on the frequencies with which letters occur in English words could probably guess a user-selected password much more quickly than was previously thought. “Attackers often use graphics processors to distribute the problem,” Duffy says. “You’d be surprised at how quickly you can guess stuff.”

In their Asilomar paper, the researchers apply the same type of mathematical analysis in a slightly different way. They consider the case in which an attacker is, from a distance, able to make a “noisy” measurement of the password stored on a credit card with an embedded chip or a key card used in a keyless-entry system.

“Noise” is the engineer’s term for anything that degrades an electromagnetic signal — such as physical obstructions, out-of-phase reflections or other electromagnetic interference. Noise comes in lots of different varieties: The familiar white noise of sleep aids is one, but so is pink noise, black noise and more exotic-sounding types of noise, such as power-law noise or Poisson noise.

In this case, rather than prior knowledge about the statistical frequency of the symbols used in a password, the attacker has prior knowledge about the probable noise characteristics of the environment: Phase noise with one set of parameters is more probable than phase noise with another set of parameters, which in turn is more probable than Brownian noise, and so on. Armed with these statistics, an attacker could infer the password stored on the card much more rapidly than was previously thought.

“Some of the approximations that we’re used to making, they make perfect sense in the context of traditional communication,” says Matthieu Bloch, an assistant professor of electrical and computer engineering at the Georgia Institute of Technology. “You design your system in a framework, and then you test it. But for crypto, you’re actually trying to prove that it’s robust to things you cannot test. So you have to be sure that your assumptions make sense from the beginning. And I think that going back to the assumptions is something people don’t do often enough.”

Bloch doubts that the failure of the uniformity assumption means that cryptographic systems in wide use today are fundamentally insecure. “My guess is that it will show that some of them are slightly less secure than we had hoped, but usually in the process, we’ll also figure out a way of patching them,” he says. The MIT and NUI researchers’ work, he says, “is very constructive, because it’s essentially saying, ‘Hey, we have to be careful.’ But it also provides a methodology to go back and reanalyze all these things.”
http://www.mit.edu/newsoffice/2013/encryption-is-less-secure-than-we-thought-0814.html
Title: Re: Encryption is less secure than we thought
Post by: Psyche on August 15, 2013, 05:53 pm
So what I got out of this is that if they can find one similarity between the encrypted and unencrypted file then it can be more easily mathematically derived...can someone cryptographically educated make a tl;dr version for dummies edition?
Title: Re: Encryption is less secure than we thought
Post by: BizittyNikkelz on August 15, 2013, 06:35 pm
Yes please dumb this down a little for me too if you can.

I never trusted encryption really... i always thought there'd be a way around, and it was just some fancy shmancy shit for the time being.

But I suppose I'll have to wait to hear the TLDR lol
Title: Re: Encryption is less secure than we thought
Post by: kmfkewm on August 16, 2013, 02:11 pm
Honestly this article is not really saying much and seems like the person who wrote it probably doesn't have a good understanding of the subject. It isn't like a wireless car ignition system transmits a password, they use zero knowledge proof of knowledge and allow for a total interception. The most I can get out of this article is that some new understanding of entropy has been arrived at, and it probably means that non-randomly generated passwords are easier to crack than previously thought, and perhaps that some cryptographic systems are also easier to directly break than was previously thought. I would need to read the paper and not this article to see what is up though, because the article doesn't really do a very good job of giving me the raw information.
Title: Re: Encryption is less secure than we thought
Post by: kmfkewm on August 16, 2013, 02:41 pm
I looked at the paper, it is multiple pages of advanced math with some English sprinkled in. I cannot make much sense of the paper unfortunately. Here is a comment from stackexchange

Quote
So the article is fluff, the details can be found in the linked paper. The just of it is a refutation of the following assertion: if you have a set of symbols chosen with identical independent distributions and subject to some kind of coding, the result can be approximated as a uniform distribution.

The paper asserts, with a few citations to some examples, that this is a common cryptographic assumption. It is, as far as I can tell from reading the literature and talking to other practitioners, not a common assumption at all. In fact, in standard encryption systems, we assume that the plaintext is chosen with a known distribution that can be arbitrary(indeed, attacker chosen), and keys are chosen randomly.

In practice, keys are not chosen randomly, they are chosen using cryptographically secure random number generators. Those can fail, but not in the way the paper is talking about.

Certain papers, such as maybe linked one about biometrics and the other about passwords, might make this erroneous assumption, but it's not common and certainly doesn't relate to what most non-practitioners would consider "encryption."

A better title for the article: is a few cryptographers made some dumb mistakes. Mistakes neither pervasive or of massive consequence.

That sounds pretty correct to me. I think this research says more about advances in cracking non-random passwords than it does anything else. They don't even mention pretty much anything the article talks about in the paper (such as keyless ignition).
Title: Re: Encryption is less secure than we thought
Post by: kybzmsrf on August 16, 2013, 04:12 pm
I never trusted encryption really... i always thought there'd be a way around

There will always be a way around because the decryption algorithms have to be publicly known and you can just start guessing the key. So making encryption secure is "just" about making keys enormously unlikely to guess right. However guessing the right key will always only be a matter of time and a little luck. Because obviously you can guess the key right in the very first try if you're lucky or in the very last of possible tries if you're not.
Today's encryption algorithms are not theoretically secure. Yet guessing the right key is so incredibly unlikely that (even with the most advanced computing technology available) it would take you years to guess a single key right. With a home PC it would take until the end of times. On average of course. That's why you call certain encryption algorithms "practically secure" although they all are theoretically insecure.

Quote
can someone cryptographically educated make a tl;dr version for dummies edition?

Not really cryptographically educated but I'll try. I'm not 100% sure if I got it right, because what they say really doesn't seem like an epiphany to me.

Imagine you have a box filled with red, blue and yellow marbles. You take any five marbles out of the box without looking at them and arrange them in a line. Let's say it's red, blue, blue, yellow, blue. That's your password or private key or whatever! Now another one wants to guess that combination of marbles.
Now that person could assume that the marbles in the box were arranged totally randomly and unpredictably when you took some out and thus every possible combination of colors of the 5 was equally likely to appear.
Thing is: When I (a random number generator) put the marbles in the box and shake the box to mix them, I'll always do that in a similar way although it seems to be random. So it might end up being the case that there's always a couple more blue marbles on top and that you are more likely to grab blue ones.
What the article (very) basically says is that "traditionally" the prior (marbles perfectly spread) was assumed and that when assuming the latter you can speed things up by guessing combinations that have a lot of blue marbles in them first because those are more likely to be right.

I find that rather obvious and thus am sure that I didn't totally get the point of the article...

The person shaking the box would be a random number generator that's used to get numbers to generate e.g. a key-pair from. Afaik it's controversial if it is even possible to generate numbers (shaking the box) perfectly randomly so that there will be no pattern at all.

Generating random numbers without having patterns appear is a huge deal and a major problem in cryptography. That's why when creating PGP-keys or when creating a truecrypt container you are asked to randomly move your cursor around in the window. And still: Because the physiology of people's hands is very similar they all move their mouses in a similar (supposedly random) way. There's your pattern again ;)

So like making encryption keys harder and harder to guess, generating random numbers is just about making the pattern more and more obscure.


Mhh... Not sure if this cleared things up even a little... ^^
Title: Re: Encryption is less secure than we thought
Post by: kmfkewm on August 16, 2013, 04:38 pm
Quote
There will always be a way around because the decryption algorithms have to be publicly known and you can just start guessing the key. So making encryption secure is "just" about making keys enormously unlikely to guess right. However guessing the right key will always only be a matter of time and a little luck. Because obviously you can guess the key right in the very first try if you're lucky or in the very last of possible tries if you're not.
Today's encryption algorithms are not theoretically secure. Yet guessing the right key is so incredibly unlikely that (even with the most advanced computing technology available) it would take you years to guess a single key right. With a home PC it would take until the end of times. On average of course. That's why you call certain encryption algorithms "practically secure" although they all are theoretically insecure.

Information Theoretically Secure: Cannot be broken, period
Computationally Secure: Cannot be broken without computational power that hopefully nobody has

Quote
What the article (very) basically says is that "traditionally" the prior (marbles perfectly spread) was assumed and that when assuming the latter you can speed things up by guessing combinations that have a lot of blue marbles in them first because those are more likely to be right.

I find that rather obvious and thus am sure that I didn't totally get the point of the article...

That is kind of what I got from it as well, but RNG's and even PRNG's should have so much randomness (or produce only randomness in the case of RNG) that this is not feasible. Also, my understanding is that all modern ciphers are protected from chosen plaintext attacks, so the randomness of the plaintext should not matter. To the best of my understanding this attack only really matters against non-randomly generated passwords and poorly designed ciphers, but yeah the math is beyond me so I could be way off here.

Quote
The person shaking the box would be a random number generator that's used to get numbers to generate e.g. a key-pair from. Afaik it's controversial if it is even possible to generate numbers (shaking the box) perfectly randomly so that there will be no pattern at all.

It is indeed debated. Some people think that true randomness is not real and that all things are completely deterministic. I think quantum physics indicates that there is randomness though, again this stuff is a bit beyond my ability to fully comprehend so take what I say for what it is worth.

Quote
Generating random numbers without having patterns appear is a huge deal and a major problem in cryptography. That's why when creating PGP-keys or when creating a truecrypt container you are asked to randomly move your cursor around in the window. And still: Because the physiology of people's hands is very similar they all move their mouses in a similar (supposedly random) way. There's your pattern again ;)

I was just thinking about this the other day. I bet when most people are asked to type randomly on their keyboards it looks something like this: eijfwoejfiewjfioewjfijfiwejfiwjfiwjfijfwejfiwejiofwejifwejfjewiofwj , and that when most people are asked to move their mouse randomly they move it in a circular pattern or up and down pattern. I think it would be better if they were asked to type a short story or to draw a random picture.
Title: Re: Encryption is less secure than we thought
Post by: kybzmsrf on August 16, 2013, 05:04 pm
I think quantum physics indicates that there is randomness though

Or it's just beyond us... Like how an entire universe can spontaneously emerge out of nothing... Weird shit...

A Geiger counter would make an excellent random number "generator"! More of a random number recorder ;)