SHA hash functions

SHA-1 (as well as SHA-0) produces a 160-bit digest from a message with a maximum length of (2⁶⁴ − 1) bits. SHA-1 is based on principles similar to those used by Ronald L. Rivest of MIT in the design of the MD4 and MD5 message digest algorithms, but has a more conservative design.

The original specification of the algorithm was published in 1993 as the Secure Hash Standard, FIPS PUB 180, by US government standards agency NIST (National Institute of Standards and Technology). This version is now often referred to as SHA-0. It was withdrawn by NSA shortly after publication and was superseded by the revised version, published in 1995 in FIPS PUB 180-1 and commonly referred to as SHA-1. SHA-1 differs from SHA-0 only by a single bitwise rotation in the message schedule of its compression function; this was done, according to NSA, to correct a flaw in the original algorithm which reduced its cryptographic security. However, NSA did not provide any further explanation or identify the flaw that was corrected. Weaknesses have subsequently been reported in both SHA-0 and SHA-1. SHA-1 appears to provide greater resistance to attacks, supporting the NSA’s assertion that the change increased the security.

SHA-2 family

The algorithms were first published in 2001 in the draft FIPS PUB 180-2, at which time review and comment were accepted. FIPS PUB 180-2, which also includes SHA-1, was released as an official standard in 2002. In February 2004, a change notice was published for FIPS PUB 180-2, specifying an additional variant, SHA-224, defined to match the key length of two-key Triple DES. These variants are patented in . The United States has released the patent under a royalty free license.

SHA-256 and SHA-512 are novel hash functions computed with 32- and 64-bit words, respectively. They use different shift amounts and additive constants, but their structures are otherwise virtually identical, differing only in the number of rounds. SHA-224 and SHA-384 are simply truncated versions of the first two, computed with different initial values.

Unlike SHA-1, the SHA-2 functions are not widely used, despite their better security. Reasons might include lack of support for SHA-2 on systems running Windows XP SP2 or older,Microsoft Corporation, [http://download.microsoft.com/download/6/8/7/687484ed-8174-496d-8db9-f02b40c12982/Overview%20of%20Windows%20XP%20Service%20Pack%203.pdf Overview of Windows XP Service Pack 3] a lack of perceived urgency since SHA-1 collisions have not yet been found, or a desire to wait until SHA-3 is standardized. SHA-256 is used to authenticate Debian Linux software packages[http://google.com/codesearch/p?hl=en#nywQboHfkw4/apt/apt-pkg/acquire-item.cc&q=SHA256 Debian codebase in Google Code] and in the DKIM message signing standard; SHA-512 is part of a system to authenticate archival video from the International Criminal Tribunal of the Rwandan genocide.John Markoff, »A Tool to Verify Digital Records, Even as Technology Shifts, New York Times, January 26, 2009 SHA-256 and SHA-512 are proposed for use in DNSSEC.RFC 5702,»RFC-Editor.org Unix and Linux vendors are moving to using 256- and 512-bit SHA-2 for secure password hashing.Ulrich Drepper, »Unix crypt with SHA-256/512 NIST's directive that U.S. government agencies stop most uses of SHA-1 after 2010, and the completion of SHA-3, may accelerate migration away from SHA-1.

Currently, the best public attacks break 41 of the 64 rounds of SHA-256 or 46 of the 80 rounds of SHA-512, as discussed in the "Cryptanalysis and Validation" section below.Yu Sasaki, Lei Wang, and Kazumaro Aoki, »Preimage Attacks on 41-Step SHA-256 and 46-Step SHA-512, accessed 3 Jan 2010

SHA-3 (in development)

An open competition for a new SHA-3 function was formally announced in the Federal Register on November 2, 2007.»National Institute of Standards and Technology "NIST is initiating an effort to develop one or more additional hash algorithms through a public competition, similar to the development process for the Advanced Encryption Standard (AES)."»CSRC hash workshop Submissions were due October 31, 2008 and the proclamation of a winner and publication of the new standard are scheduled to take place in 2012.

Comparison of SHA functions

{| border=1 class="wikitable" ! colspan="2" | Algorithm and
variant !Output size (bits) !Internal state size (bits) !Block size (bits) !Max message size (bits) !Word size (bits) !Rounds !Operations !Collisions found |- align="center" | colspan="2" | SHA-0 || 160 || 160 || 512 || 2⁶⁴ − 1 || 32 || 80 || ,and,or,xor,rot || Yes |- align="center" | colspan="2" | SHA-1 || 160 || 160 || 512 || 2⁶⁴ − 1 || 32 || 80 || ,and,or,xor,rot || None (2⁶³ attack)»A new 2⁵² attack on SHA-1, Retrieved on 2009-05-08. |- align="center" | rowspan="2" | SHA-2 || SHA-256/224 || 256/224 || 256 || 512 || 2⁶⁴ − 1 || 32 || 64 || ,and,or,xor,shr,rot || None |- align="center" | SHA-512/384 || 512/384 || 512 || 1024 || 2¹²⁸ − 1 || 64 || 80 || ,and,or,xor,shr,rot || None |}

Applications

SHA-1 is the most widely employed of the SHA family. It forms part of several widely-used security applications and protocols, including TLS and SSL, PGP, SSH, S/MIME, and IPsec. Those applications can also use MD5; both MD5 and SHA-1 are descended from MD4. SHA-1 hashing is also used in distributed revision control systems such as Git, Mercurial, and Monotone to identify revisions, and to detect data corruption or tampering.

SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512 are the secure hash algorithms required by law for use in certain U. S. Government applications, including use within other cryptographic algorithms and protocols, for the protection of sensitive unclassified information. FIPS PUB 180-1 also encouraged adoption and use of SHA-1 by private and commercial organizations. SHA-1 is being retired for most government uses; the U.S. National Institute of Standards and Technology says, "Federal agencies should stop using SHA-1 for...applications that require collision resistance as soon as practical, and must use the SHA-2 family of hash functions for these applications after 2010" (emphasis in original).National Institute on Standards and Technology Computer Security Resource Center, »NIST's Policy on Hash Functions, accessed March 29, 2009.

A prime motivation for the publication of the Secure Hash Algorithm was the Digital Signature Standard, in which it is incorporated.

The SHA hash functions have been used as the basis for the SHACAL block ciphers.

Cryptanalysis and validation

Cryptographers have produced collision pairs for SHA-0 and have found algorithms that should produce SHA-1 collisions in far fewer than the originally expected 2⁸⁰ evaluations.

In terms of practical security, a major concern about these new attacks is that they might pave the way to more efficient ones. Whether this is the case has yet to be seen, but a migration to stronger hashes is believed to be prudent. Some of the applications that use cryptographic hashes, such as password storage, are only minimally affected by a collision attack. Constructing a password that works for a given account requires a preimage attack, as well as access to the hash of the original password (typically in the shadow file) which may or may not be trivial. Reversing password encryption (e.g. to obtain a password to try against a user's account elsewhere) is not made possible by the attacks. (However, even a secure password hash can't prevent brute-force attacks on weak passwords.)

Official validation

Implementations of all FIPS-approved security functions can be officially validated through the CMVP program, jointly run by the National Institute of Standards and Technology (NIST) and the Communications Security Establishment (CSE). For informal verification, a package to generate a high number of test vectors is made available for download on the NIST site; the resulting verification however does not replace in any way the formal CMVP validation, which is required by law for certain applications.

, there are more than 500 validated implementations of SHA-1, with fewer than ten of them capable of handling messages with a length in bits not a multiple of eight (see »SHS Validation List).

Examples and pseudocode

The following is an example of SHA-1 digests. ASCII encoding is assumed for all messages.

,SHA1("The quick brown fox jumps over the lazy dog") ,= 2fd4e1c6 7a2d28fc ed849ee1 bb76e739 1b93eb12

Even a small change in the message will, with overwhelming probability, result in a completely different hash due to the avalanche effect. For example, changing dog to cog produces a hash with different values for 81 of the 160 bits:

,SHA1("The quick brown fox jumps over the lazy c'''og") ,= de9f2c7f d25e1b3a fad3e85a 0bd17d9b 100db4b3

SHA-1 pseudocode

,''Note 1: All variables are unsigned 32 bits and wrap modulo 2³² when calculating ,Note 2: All constants in this pseudo code are in big endian. , Within each word, the most significant byte is stored in the leftmost byte position , ,Initialize variables: ,h0 = 0x67452301 ,h1 = 0xEFCDAB89 ,h2 = 0x98BADCFE ,h3 = 0x10325476 ,h4 = 0xC3D2E1F0 , ,Pre-processing: ,append the bit '1' to the message ,append 0 ≤ k < 512 bits '0', so that the resulting message length (in bits) , is congruent to 448 ≡ −64 (mod 512) ,append length of message (before pre-processing), in bits, as 64-bit big-endian integer , ,Process the message in successive 512-bit chunks: ,break message into 512-bit chunks ,for each chunk , break chunk into sixteen 32-bit big-endian words w[i], 0 ≤ i ≤ 15 , , Extend the sixteen 32-bit words into eighty 32-bit words: , for i from 16 to 79 , w[i] = (w[i-3] xor w[i-8] xor w[i-14] xor w[i-16]) leftrotate 1 , , Initialize hash value for this chunk: , a = h0 , b = h1 , c = h2 , d = h3 , e = h4 , , Main loop: , for i from 0 to 79 , if 0 ≤ i ≤ 19 then , f = (b and c) or ((not b) and d) , k = 0x5A827999 , else if 20 ≤ i ≤ 39 , f = b xor c xor d , k = 0x6ED9EBA1 , else if 40 ≤ i ≤ 59 , f = (b and c) or (b and d) or (c and d) , k = 0x8F1BBCDC , else if 60 ≤ i ≤ 79 , f = b xor c xor d , k = 0xCA62C1D6 , , temp = (a leftrotate 5) + f + e + k + w[i] , e = d , d = c , c = b leftrotate 30 , b = a , a = temp , , Add this chunk's hash to result so far: , h0 = h0 + a , h1 = h1 + b , h2 = h2 + c , h3 = h3 + d , h4 = h4 + e , ,Produce the final hash value (big-endian): ,digest = hash = h0 append h1 append h2 append h3 append h4

The constant values used are chosen as nothing up my sleeve numbers: the four round constants k are 2³⁰ times the square roots of 2, 3, 5 and 10 The first four starting values for h0 through h3 are the same as the MD5 algorithm, and the fifth (for h4) is similar.

Instead of the formulation from the original FIPS PUB 180-1 shown, the following equivalent expressions may be used to compute f in the main loop above:

,(0 ≤ i ≤ 19): f = d xor (b and (c xor d)) (alternative 1) ,(0 ≤ i ≤ 19): f = (b and c) xor ((not b) and d) (alternative 2) ,(0 ≤ i ≤ 19): f = (b and c) + ((not b) and d) (alternative 3) ,(0 ≤ i ≤ 19): f = vec_sel(d, c, b) (alternative 4) ,  ,(40 ≤ i ≤ 59): f = (b and c) or (d and (b or c)) (alternative 1) ,(40 ≤ i ≤ 59): f = (b and c) or (d and (b xor c)) (alternative 2) ,(40 ≤ i ≤ 59): f = (b and c) + (d and (b xor c)) (alternative 3) ,(40 ≤ i ≤ 59): f = (b and c) xor (b and d) xor (c and d) (alternative 4)

SHA-256 (a SHA-2 variant) pseudocode

,Note 1: All variables are unsigned 32 bits and wrap modulo 2³² when calculating ,Note 2: All constants in this pseudo code are in big endian , ,''Initialize variables ,(first 32 bits of the fractional parts of the square roots of the first 8 primes 2..19): ,h0 := 0x6a09e667 ,h1 := 0xbb67ae85 ,h2 := 0x3c6ef372 ,h3 := 0xa54ff53a ,h4 := 0x510e527f ,h5 := 0x9b05688c ,h6 := 0x1f83d9ab ,h7 := 0x5be0cd19 , ,Initialize table of round constants ,(first 32 bits of the fractional parts of the cube roots of the first 64 primes 2..311): ,k[0..63] := , 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, , 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, , 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, , 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, , 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, , 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, , 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, , 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2 , ,Pre-processing: ,append the bit '1' to the message ,append k bits '0', where k is the minimum number >= 0 such that the resulting message , length (in bits) is congruent to 448 (mod 512) ,append length of message (before pre-processing), in bits, as 64-bit big-endian integer , ,Process the message in successive 512-bit chunks: ,break message into 512-bit chunks ,for each chunk , break chunk into sixteen 32-bit big-endian words w[0..15] , , Extend the sixteen 32-bit words into sixty-four 32-bit words: , for i from 16 to 63 , s0 := (w[i-15] rightrotate 7) xor (w[i-15] rightrotate 18) xor (w[i-15] rightshift 3) , s1 := (w[i-2] rightrotate 17) xor (w[i-2] rightrotate 19) xor (w[i-2] rightshift 10) , w[i] := w[i-16] s0 w[i-7] s1 , , Initialize hash value for this chunk: , a := h0 , b := h1 , c := h2 , d := h3 , e := h4 , f := h5 , g := h6 , h := h7 , , Main loop: , for i from 0 to 63 , s0 := (a rightrotate 2) xor (a rightrotate 13) xor (a rightrotate 22) , maj := (a and b) xor (a and c) xor (b and c) , t2 := s0 maj , s1 := (e rightrotate 6) xor (e rightrotate 11) xor (e rightrotate 25) , ch := (e and f) xor ((not e) and g) , t1 := h + s1 + ch + k[i] + w[i] , , h := g , g := f , f := e , e := d + t1 , d := c , c := b , b := a , a := t1 + t2 , , Add this chunk's hash to result so far: , h0 := h0 + a , h1 := h1 + b , h2 := h2 + c , h3 := h3 + d , h4 := h4 + e , h5 := h5 + f , h6 := h6 + g , h7 := h7 + h , ,Produce the final hash value (big-endian): ,digest = hash = h0 append h1 append h2 append h3 append h4 append h5 append h6 append h7

The ch and maj functions can be optimized the same way as described for SHA-1.

SHA-224 is identical to SHA-256, except that:

• the initial variable values h0 through h7 are different, and
• the output is constructed by omitting h7.