)]}'
{
  "commit": "674534a700cc110b374f9c0a6c7f518e18392af2",
  "tree": "b4d6b0cd0c792dfbec43d55f6da4c8eb976682b0",
  "parents": [
    "4156060ca498673b9938a512bc84d95a8a9c74b2"
  ],
  "author": {
    "name": "Nick Terrell",
    "email": "terrelln@fb.com",
    "time": "Fri Jun 21 15:39:43 2019 -0700"
  },
  "committer": {
    "name": "Nick Terrell",
    "email": "terrelln@fb.com",
    "time": "Fri Jun 21 15:47:31 2019 -0700"
  },
  "message": "[zstd] Fix data corruption in niche use case\n\n* Extract the overflow correction into a helper function.\n* Load the dictionary `ZSTD_CHUNKSIZE_MAX \u003d 512 MB` bytes at a time\n  and overflow correct between each chunk.\n\nData corruption could happen when all these conditions are true:\n\n* You are using multithreading mode\n* Your overlap size is \u003e\u003d 512 MB (implies window size \u003e\u003d 512 MB)\n* You are using a strategy \u003e\u003d ZSTD_btlazy\n* You are compressing more than 4 GB\n\nThe problem is that when loading a large dictionary we don\u0027t do\noverflow correction. We can only load 512 MB at a time, and may\nneed to do overflow correction before each chunk.\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "77b8b9b70cf9ac26a80f428bc67b9bccfd012edf",
      "old_mode": 33188,
      "old_path": "lib/compress/zstd_compress.c",
      "new_id": "ad4e8a40ec9cd1bd5ff1f02072866ea5f0556f3f",
      "new_mode": 33188,
      "new_path": "lib/compress/zstd_compress.c"
    }
  ]
}