问题 有效地将ByteString转换为十六进制表示


我需要能够给出SHA512哈希的十六进制表示。也许我只是看起来不够努力,但我可以在Hackage上找到任何功能来做到这一点。所以我写了一个实现使用 unfoldrN。这对我的目的来说肯定足够快,但我想知道是否有人知道更快的方法。

我把我的实现作为一个要点放在Github上: https://gist.github.com/2356925 。该文件还包括一个基于的简单实现 Numeric.showHex,QuickCheck测试和标准基准。我目前的简单版本的结果与 unfoldrN 版本是:

benchmarking simple
mean: 4.677296 ms, lb 4.656011 ms, ub 4.696684 ms, ci 0.950
std dev: 104.2791 us, lb 87.77023 us, ub 128.1627 us, ci 0.950
found 5 outliers among 100 samples (5.0%)
  4 (4.0%) low mild
variance introduced by outliers: 15.195%
variance is moderately inflated by outliers

benchmarking unfoldrN_MS1
mean: 370.0101 us, lb 365.9819 us, ub 373.8619 us, ci 0.950
std dev: 20.17016 us, lb 16.92772 us, ub 24.08982 us, ci 0.950
found 14 outliers among 100 samples (14.0%)
  7 (7.0%) low mild
  7 (7.0%) high mild
variance introduced by outliers: 52.467%
variance is severely inflated by outliers

有人想要改进它吗?


4956
2018-04-11 04:43


起源

whosawthatcoming.com/private/PMBFOGQIHT BLAM! - Will


答案:


走低级,

import Data.ByteString.Internal
import Foreign.Ptr
import Foreign.Storable
import qualified Data.ByteString as B
import Data.ByteString.Unsafe
import Data.Bits
import Data.Word

maxLen :: Int
maxLen = maxBound `quot` 2

hexDig :: Word8 -> Word8
hexDig d
    | d < 10    = d + 48
    | otherwise = d + 87

toHex :: ByteString -> ByteString
toHex bs
    | len > maxLen = error "too long to convert"
    | otherwise    = unsafeCreate nl (go 0)
      where
        len = B.length bs
        nl  = 2*len
        go i p
            | i == len  = return ()
            | otherwise = case unsafeIndex bs i of
                            w -> do poke p (hexDig $ w `shiftR` 4)
                                    poke (p `plusPtr` 1) (hexDig $ w .&. 0xF)
                                    go (i+1) (p `plusPtr` 2)

我可以将我的盒子上的转换时间减少大约3.5倍。制造 sample 我得到了更长的时间(25000)

benchmarking simple
mean: 13.76532 ms, lb 13.64184 ms, ub 13.88680 ms, ci 0.950
std dev: 633.2413 us, lb 582.6342 us, ub 687.9701 us, ci 0.950
variance introduced by outliers: 44.438%
variance is moderately inflated by outliers

benchmarking unfoldrN_MS1
mean: 430.5705 us, lb 424.9206 us, ub 438.5689 us, ci 0.950
std dev: 33.85429 us, lb 26.25623 us, ub 45.74915 us, ci 0.950
found 4 outliers among 100 samples (4.0%)
  3 (3.0%) high mild
  1 (1.0%) high severe
variance introduced by outliers: 69.726%
variance is severely inflated by outliers

benchmarking LowHex
mean: 123.6000 us, lb 123.0551 us, ub 124.7084 us, ci 0.950
std dev: 3.837497 us, lb 1.869370 us, ub 6.470112 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
  4 (4.0%) high mild
  2 (2.0%) high severe
variance introduced by outliers: 25.818%
variance is moderately inflated by outliers

对于原来的500长 sample, 它是

benchmarking simple
mean: 2.603306 ms, lb 2.583054 ms, ub 2.629212 ms, ci 0.950
std dev: 116.5341 us, lb 81.61409 us, ub 191.3293 us, ci 0.950
found 7 outliers among 100 samples (7.0%)
  2 (2.0%) low severe
  3 (3.0%) low mild
  1 (1.0%) high severe
variance introduced by outliers: 42.490%
variance is moderately inflated by outliers

benchmarking unfoldrN_MS1
mean: 83.19349 us, lb 82.88474 us, ub 83.58283 us, ci 0.950
std dev: 1.771460 us, lb 1.486104 us, ub 2.174729 us, ci 0.950
found 14 outliers among 100 samples (14.0%)
  12 (12.0%) high mild
  2 (2.0%) high severe
variance introduced by outliers: 14.225%
variance is moderately inflated by outliers

benchmarking LowHex
mean: 24.50564 us, lb 24.41683 us, ub 24.61241 us, ci 0.950
std dev: 497.1908 ns, lb 415.6366 ns, ub 609.7594 ns, ci 0.950
found 5 outliers among 100 samples (5.0%)
  5 (5.0%) high mild
variance introduced by outliers: 13.256%
variance is moderately inflated by outliers

8
2018-04-11 09:55



不错的加速。它看起来似乎没有更快的速度,所以我猜这个赢了。你能想到添加这个功能有意义的任何包吗? - Michael Snoyman


你正在寻找的功能是 Data.ByteString.Builder.byteStringHex (或它的Lazy ByteStrings的双功能),由新的字节串构建器提供。一世 扩展了你的基准 并在我的机器上获得以下结果:

benchmarking size 5000/simple
mean: 2.469847 ms, lb 2.440422 ms, ub 2.522850 ms, ci 0.950
std dev: 196.5903 us, lb 116.8811 us, ub 318.4720 us, ci 0.950
found 16 outliers among 100 samples (16.0%)
  3 (3.0%) low severe
  2 (2.0%) low mild
  10 (10.0%) high severe
variance introduced by outliers: 70.721%
variance is severely inflated by outliers

benchmarking size 5000/unfoldrN_MS1
mean: 102.6075 us, lb 101.7695 us, ub 104.0159 us, ci 0.950
std dev: 5.468574 us, lb 3.681120 us, ub 8.080665 us, ci 0.950
found 16 outliers among 100 samples (16.0%)
  6 (6.0%) high mild
  10 (10.0%) high severe
variance introduced by outliers: 51.455%
variance is severely inflated by outliers

benchmarking size 5000/byteStringHexFixed
mean: 5.675204 us, lb 5.636296 us, ub 5.750211 us, ci 0.950
std dev: 264.3726 ns, lb 140.9738 ns, ub 398.8494 ns, ci 0.950
found 5 outliers among 100 samples (5.0%)
  4 (4.0%) high severe
variance introduced by outliers: 44.476%
variance is moderately inflated by outliers

我喜欢这个号码。太糟糕了,Duncan Coutts仍在审查我对字节串库的补丁。最迟,新的字节字符串构建器将在下一个GHC版本中提供。


6
2018-04-25 08:13





看起来我刚刚使用过

hex :: B.ByteString -> String
hex = concatMap (printf "%02x") . B.unpack

上次我想这样做。这与Crypto.Hash库iirc一起使用。我怀疑性能是那么好,但与(慢)sha512函数本身相比,为什么十六进制转换会成为问题?


2
2018-04-19 05:35