mirror of https://github.com/gchq/CyberChef.git
Bring up to date with master
This commit is contained in:
j433866
commit
dd5038c15b
|
|
@ -1,14 +1 @@
|
|||
<!-- Prefix the title above with one of the following: -->
|
||||
<!-- Bug report: -->
|
||||
<!-- Operation request: -->
|
||||
<!-- Feature request: -->
|
||||
<!-- Misc: -->
|
||||
|
||||
### Summary
|
||||
|
||||
|
||||
### Example
|
||||
<!-- If describing a bug, tell us what happens instead of the expected behavior -->
|
||||
<!-- Include a link that triggers the bug if possible -->
|
||||
<!-- If you are requesting a new operation, include example input and output -->
|
||||
|
||||
<!-- Prefix the title above with 'Misc:' -->
|
||||
|
|
|
|||
|
|
@ -0,0 +1,35 @@
|
|||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve
|
||||
title: 'Bug report: <Insert title here>'
|
||||
labels: bug
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
<!-- Prefix the title above with 'Bug report:' -->
|
||||
**Describe the bug**
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
**To Reproduce**
|
||||
Steps to reproduce the behavior or a link to the recipe / input used to cause the bug:
|
||||
|
||||
1. Go to '...'
|
||||
2. Click on '....'
|
||||
3. Scroll down to '....'
|
||||
4. See error
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen.
|
||||
|
||||
**Screenshots**
|
||||
If applicable, add screenshots to help explain your problem.
|
||||
|
||||
**Desktop (if relevant, please complete the following information):**
|
||||
- OS: [e.g. Windows]
|
||||
- Browser [e.g. chrome, safari]
|
||||
- Version [e.g. 22]
|
||||
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
---
|
||||
name: Feature request
|
||||
about: Suggest an idea for the project
|
||||
title: 'Feature request: <Insert title here>'
|
||||
labels: feature
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
<!-- Prefix the title above with 'Feature request:' -->
|
||||
**Is your feature request related to a problem? Please describe.**
|
||||
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
|
||||
|
||||
**Describe the solution you'd like**
|
||||
A clear and concise description of what you want to happen.
|
||||
|
||||
**Describe alternatives you've considered**
|
||||
A clear and concise description of any alternative solutions or features you've considered.
|
||||
|
||||
**Additional context**
|
||||
Add any other context or screenshots about the feature request here.
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
name: Operation request
|
||||
about: Suggest a new operation
|
||||
title: 'Operation request: <Insert title here>'
|
||||
labels: operation
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
<!-- Prefix the title above with 'Operation request:' -->
|
||||
|
||||
## Summary
|
||||
|
||||
### Example Input
|
||||
|
||||
### Example Output
|
||||
18
CHANGELOG.md
18
CHANGELOG.md
|
|
@ -2,6 +2,16 @@
|
|||
All major and minor version changes will be documented in this file. Details of patch-level version changes can be found in [commit messages](https://github.com/gchq/CyberChef/commits/master).
|
||||
|
||||
|
||||
### [8.34.0] - 2019-06-28
|
||||
- Various new visualisations added to the 'Entropy' operation [@MShwed] | [#535]
|
||||
- Efficiency improvements made to the 'Entropy' operation for large file support [@n1474335]
|
||||
|
||||
### [8.33.0] - 2019-06-27
|
||||
- 'Bzip2 Compress' operation added and 'Bzip2 Decompress' operation greatly improved [@artemisbot] | [#531]
|
||||
|
||||
### [8.32.0] - 2019-06-27
|
||||
- 'Indec of Coincidence' operation added [@Ge0rg3] | [#571]
|
||||
|
||||
### [8.31.0] - 2019-04-12
|
||||
- The downloadable version of CyberChef is now a .zip file containing separate modules rather than a single .htm file. It is still completely standalone and will not make any external network requests. This change reduces the complexity of the build process significantly. [@n1474335]
|
||||
|
||||
|
|
@ -130,6 +140,9 @@ All major and minor version changes will be documented in this file. Details of
|
|||
|
||||
|
||||
|
||||
[8.34.0]: https://github.com/gchq/CyberChef/releases/tag/v8.34.0
|
||||
[8.33.0]: https://github.com/gchq/CyberChef/releases/tag/v8.33.0
|
||||
[8.32.0]: https://github.com/gchq/CyberChef/releases/tag/v8.32.0
|
||||
[8.31.0]: https://github.com/gchq/CyberChef/releases/tag/v8.31.0
|
||||
[8.30.0]: https://github.com/gchq/CyberChef/releases/tag/v8.30.0
|
||||
[8.29.0]: https://github.com/gchq/CyberChef/releases/tag/v8.29.0
|
||||
|
|
@ -189,6 +202,8 @@ All major and minor version changes will be documented in this file. Details of
|
|||
[@Cynser]: https://github.com/Cynser
|
||||
[@anthony-arnold]: https://github.com/anthony-arnold
|
||||
[@masq]: https://github.com/masq
|
||||
[@Ge0rg3]: https://github.com/Ge0rg3
|
||||
[@MShwed]: https://github.com/MShwed
|
||||
|
||||
[#95]: https://github.com/gchq/CyberChef/pull/299
|
||||
[#173]: https://github.com/gchq/CyberChef/pull/173
|
||||
|
|
@ -231,4 +246,7 @@ All major and minor version changes will be documented in this file. Details of
|
|||
[#506]: https://github.com/gchq/CyberChef/pull/506
|
||||
[#516]: https://github.com/gchq/CyberChef/pull/516
|
||||
[#525]: https://github.com/gchq/CyberChef/pull/525
|
||||
[#531]: https://github.com/gchq/CyberChef/pull/531
|
||||
[#533]: https://github.com/gchq/CyberChef/pull/533
|
||||
[#535]: https://github.com/gchq/CyberChef/pull/535
|
||||
[#571]: https://github.com/gchq/CyberChef/pull/571
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
75
package.json
75
package.json
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "cyberchef",
|
||||
"version": "8.31.6",
|
||||
"version": "8.34.2",
|
||||
"description": "The Cyber Swiss Army Knife for encryption, encoding, compression and data analysis.",
|
||||
"author": "n1474335 <n1474335@gmail.com>",
|
||||
"homepage": "https://gchq.github.io/CyberChef",
|
||||
|
|
@ -29,29 +29,35 @@
|
|||
},
|
||||
"main": "build/node/CyberChef.js",
|
||||
"bugs": "https://github.com/gchq/CyberChef/issues",
|
||||
"browserslist": [
|
||||
"Chrome >= 40",
|
||||
"Firefox >= 35",
|
||||
"Edge >= 14",
|
||||
"node >= 6.5"
|
||||
],
|
||||
"devDependencies": {
|
||||
"@babel/core": "^7.4.4",
|
||||
"@babel/core": "^7.4.5",
|
||||
"@babel/plugin-transform-runtime": "^7.4.4",
|
||||
"@babel/preset-env": "^7.4.4",
|
||||
"autoprefixer": "^9.5.1",
|
||||
"babel-eslint": "^10.0.1",
|
||||
"babel-loader": "^8.0.5",
|
||||
"@babel/preset-env": "^7.4.5",
|
||||
"autoprefixer": "^9.6.0",
|
||||
"babel-eslint": "^10.0.2",
|
||||
"babel-loader": "^8.0.6",
|
||||
"babel-plugin-syntax-dynamic-import": "^6.18.0",
|
||||
"chromedriver": "^74.0.0",
|
||||
"chromedriver": "^75.0.0",
|
||||
"colors": "^1.3.3",
|
||||
"css-loader": "^2.1.1",
|
||||
"eslint": "^5.16.0",
|
||||
"css-loader": "^3.0.0",
|
||||
"eslint": "^6.0.1",
|
||||
"exports-loader": "^0.7.0",
|
||||
"file-loader": "^3.0.1",
|
||||
"file-loader": "^4.0.0",
|
||||
"grunt": "^1.0.4",
|
||||
"grunt-accessibility": "~6.0.0",
|
||||
"grunt-chmod": "~1.1.1",
|
||||
"grunt-concurrent": "^2.3.1",
|
||||
"grunt-concurrent": "^3.0.0",
|
||||
"grunt-contrib-clean": "~2.0.0",
|
||||
"grunt-contrib-connect": "^2.0.0",
|
||||
"grunt-contrib-copy": "~1.0.0",
|
||||
"grunt-contrib-watch": "^1.1.0",
|
||||
"grunt-eslint": "^21.0.0",
|
||||
"grunt-eslint": "^21.1.0",
|
||||
"grunt-exec": "~3.0.0",
|
||||
"grunt-jsdoc": "^2.4.0",
|
||||
"grunt-webpack": "^3.1.3",
|
||||
|
|
@ -60,43 +66,43 @@
|
|||
"imports-loader": "^0.8.0",
|
||||
"ink-docstrap": "^1.3.2",
|
||||
"jsdoc-babel": "^0.5.0",
|
||||
"mini-css-extract-plugin": "^0.6.0",
|
||||
"nightwatch": "^1.0.19",
|
||||
"mini-css-extract-plugin": "^0.7.0",
|
||||
"nightwatch": "^1.1.12",
|
||||
"node-sass": "^4.12.0",
|
||||
"postcss-css-variables": "^0.12.0",
|
||||
"postcss-css-variables": "^0.13.0",
|
||||
"postcss-import": "^12.0.1",
|
||||
"postcss-loader": "^3.0.0",
|
||||
"prompt": "^1.0.0",
|
||||
"sass-loader": "^7.1.0",
|
||||
"sitemap": "^2.2.0",
|
||||
"style-loader": "^0.23.1",
|
||||
"svg-url-loader": "^2.3.2",
|
||||
"url-loader": "^1.1.2",
|
||||
"webpack": "^4.31.0",
|
||||
"svg-url-loader": "^2.3.3",
|
||||
"url-loader": "^2.0.1",
|
||||
"webpack": "^4.35.0",
|
||||
"webpack-bundle-analyzer": "^3.3.2",
|
||||
"webpack-dev-server": "^3.3.1",
|
||||
"webpack-dev-server": "^3.7.2",
|
||||
"webpack-node-externals": "^1.7.2",
|
||||
"worker-loader": "^2.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@babel/polyfill": "^7.4.4",
|
||||
"@babel/runtime": "^7.4.4",
|
||||
"@babel/runtime": "^7.4.5",
|
||||
"arrive": "^2.4.1",
|
||||
"babel-plugin-transform-builtin-extend": "1.1.2",
|
||||
"bcryptjs": "^2.4.3",
|
||||
"bignumber.js": "^8.1.1",
|
||||
"bignumber.js": "^9.0.0",
|
||||
"blakejs": "^1.1.0",
|
||||
"bootstrap": "4.2.1",
|
||||
"bootstrap": "4.3.1",
|
||||
"bootstrap-colorpicker": "^2.5.3",
|
||||
"bootstrap-material-design": "^4.1.1",
|
||||
"bootstrap-material-design": "^4.1.2",
|
||||
"bson": "^4.0.2",
|
||||
"chi-squared": "^1.1.0",
|
||||
"clippyjs": "0.0.3",
|
||||
"core-js": "^3.0.1",
|
||||
"core-js": "^3.1.4",
|
||||
"crypto-api": "^0.8.3",
|
||||
"crypto-js": "^3.1.9-1",
|
||||
"ctph.js": "0.0.5",
|
||||
"d3": "^5.9.2",
|
||||
"d3": "^5.9.4",
|
||||
"d3-hexbin": "^0.2.2",
|
||||
"diff": "^4.0.1",
|
||||
"es6-promisify": "^6.0.1",
|
||||
|
|
@ -104,27 +110,28 @@
|
|||
"esmangle": "^1.0.1",
|
||||
"esprima": "^4.0.1",
|
||||
"exif-parser": "^0.1.12",
|
||||
"file-saver": "^2.0.1",
|
||||
"file-saver": "^2.0.2",
|
||||
"geodesy": "^1.1.3",
|
||||
"highlight.js": "^9.15.6",
|
||||
"highlight.js": "^9.15.8",
|
||||
"jimp": "^0.6.4",
|
||||
"jquery": "3.4.1",
|
||||
"js-crc": "^0.2.0",
|
||||
"js-sha3": "^0.8.0",
|
||||
"jsesc": "^2.5.2",
|
||||
"jsonpath": "^1.0.1",
|
||||
"jsonpath": "^1.0.2",
|
||||
"jsonwebtoken": "^8.5.1",
|
||||
"jsqr": "^1.2.0",
|
||||
"jsrsasign": "8.0.12",
|
||||
"kbpgp": "2.1.0",
|
||||
"kbpgp": "2.1.2",
|
||||
"libbzip2-wasm": "0.0.4",
|
||||
"libyara-wasm": "0.0.12",
|
||||
"lodash": "^4.17.11",
|
||||
"loglevel": "^1.6.1",
|
||||
"loglevel": "^1.6.3",
|
||||
"loglevel-message-prefix": "^3.0.0",
|
||||
"moment": "^2.24.0",
|
||||
"moment-timezone": "^0.5.25",
|
||||
"ngeohash": "^0.6.3",
|
||||
"node-forge": "^0.8.2",
|
||||
"node-forge": "^0.8.5",
|
||||
"node-md6": "^0.1.0",
|
||||
"nodom": "^2.2.0",
|
||||
"notepack.io": "^2.2.0",
|
||||
|
|
@ -132,12 +139,12 @@
|
|||
"otp": "^0.1.3",
|
||||
"popper.js": "^1.15.0",
|
||||
"qr-image": "^3.2.0",
|
||||
"scryptsy": "^2.0.0",
|
||||
"scryptsy": "^2.1.0",
|
||||
"snackbarjs": "^1.1.0",
|
||||
"sortablejs": "^1.9.0",
|
||||
"split.js": "^1.5.10",
|
||||
"split.js": "^1.5.11",
|
||||
"ssdeep.js": "0.0.2",
|
||||
"ua-parser-js": "^0.7.19",
|
||||
"ua-parser-js": "^0.7.20",
|
||||
"utf8": "^3.0.0",
|
||||
"vkbeautify": "^0.99.3",
|
||||
"xmldom": "^0.1.27",
|
||||
|
|
|
|||
|
|
@ -1,13 +1,7 @@
|
|||
module.exports = {
|
||||
plugins: [
|
||||
require("postcss-import"),
|
||||
require("autoprefixer")({
|
||||
browsers: [
|
||||
"Chrome >= 40",
|
||||
"Firefox >= 35",
|
||||
"Edge >= 14"
|
||||
]
|
||||
}),
|
||||
require("autoprefixer"),
|
||||
require("postcss-css-variables")({
|
||||
preserve: true
|
||||
}),
|
||||
|
|
|
|||
|
|
@ -201,11 +201,20 @@ class Utils {
|
|||
* Utils.parseEscapedChars("\\n");
|
||||
*/
|
||||
static parseEscapedChars(str) {
|
||||
return str.replace(/(\\)?\\([bfnrtv0'"]|x[\da-fA-F]{2}|u[\da-fA-F]{4}|u\{[\da-fA-F]{1,6}\})/g, function(m, a, b) {
|
||||
return str.replace(/(\\)?\\([bfnrtv'"]|[0-3][0-7]{2}|[0-7]{1,2}|x[\da-fA-F]{2}|u[\da-fA-F]{4}|u\{[\da-fA-F]{1,6}\}|\\)/g, function(m, a, b) {
|
||||
if (a === "\\") return "\\"+b;
|
||||
switch (b[0]) {
|
||||
case "\\":
|
||||
return "\\";
|
||||
case "0":
|
||||
return "\0";
|
||||
case "1":
|
||||
case "2":
|
||||
case "3":
|
||||
case "4":
|
||||
case "5":
|
||||
case "6":
|
||||
case "7":
|
||||
return String.fromCharCode(parseInt(b, 8));
|
||||
case "b":
|
||||
return "\b";
|
||||
case "t":
|
||||
|
|
|
|||
|
|
@ -395,6 +395,7 @@
|
|||
"ops": [
|
||||
"Entropy",
|
||||
"Frequency distribution",
|
||||
"Index of Coincidence",
|
||||
"Chi Square",
|
||||
"Disassemble x86",
|
||||
"Pseudo-Random Number Generator",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,72 @@
|
|||
/**
|
||||
* @author Matt C [me@mitt.dev]
|
||||
* @copyright Crown Copyright 2019
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import Operation from "../Operation";
|
||||
import OperationError from "../errors/OperationError";
|
||||
import Bzip2 from "libbzip2-wasm";
|
||||
|
||||
/**
|
||||
* Bzip2 Compress operation
|
||||
*/
|
||||
class Bzip2Compress extends Operation {
|
||||
|
||||
/**
|
||||
* Bzip2Compress constructor
|
||||
*/
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
this.name = "Bzip2 Compress";
|
||||
this.module = "Compression";
|
||||
this.description = "Bzip2 is a compression library developed by Julian Seward (of GHC fame) that uses the Burrows-Wheeler algorithm. It only supports compressing single files and its compression is slow, however is more effective than Deflate (.gz & .zip).";
|
||||
this.infoURL = "https://wikipedia.org/wiki/Bzip2";
|
||||
this.inputType = "ArrayBuffer";
|
||||
this.outputType = "ArrayBuffer";
|
||||
this.args = [
|
||||
{
|
||||
name: "Block size (100s of kb)",
|
||||
type: "number",
|
||||
value: 9,
|
||||
min: 1,
|
||||
max: 9
|
||||
},
|
||||
{
|
||||
name: "Work factor",
|
||||
type: "number",
|
||||
value: 30
|
||||
}
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {ArrayBuffer} input
|
||||
* @param {Object[]} args
|
||||
* @returns {File}
|
||||
*/
|
||||
run(input, args) {
|
||||
const [blockSize, workFactor] = args;
|
||||
if (input.byteLength <= 0) {
|
||||
throw new OperationError("Please provide an input.");
|
||||
}
|
||||
if (ENVIRONMENT_IS_WORKER()) self.sendStatusMessage("Loading Bzip2...");
|
||||
return new Promise((resolve, reject) => {
|
||||
Bzip2().then(bzip2 => {
|
||||
if (ENVIRONMENT_IS_WORKER()) self.sendStatusMessage("Compressing data...");
|
||||
const inpArray = new Uint8Array(input);
|
||||
const bzip2cc = bzip2.compressBZ2(inpArray, blockSize, workFactor);
|
||||
if (bzip2cc.error !== 0) {
|
||||
reject(new OperationError(bzip2cc.error_msg));
|
||||
} else {
|
||||
const output = bzip2cc.output;
|
||||
resolve(output.buffer.slice(output.byteOffset, output.byteLength + output.byteOffset));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export default Bzip2Compress;
|
||||
|
|
@ -1,12 +1,12 @@
|
|||
/**
|
||||
* @author n1474335 [n1474335@gmail.com]
|
||||
* @copyright Crown Copyright 2016
|
||||
* @author Matt C [me@mitt.dev]
|
||||
* @copyright Crown Copyright 2019
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import Operation from "../Operation";
|
||||
import bzip2 from "../vendor/bzip2";
|
||||
import OperationError from "../errors/OperationError";
|
||||
import Bzip2 from "libbzip2-wasm";
|
||||
|
||||
/**
|
||||
* Bzip2 Decompress operation
|
||||
|
|
@ -23,9 +23,15 @@ class Bzip2Decompress extends Operation {
|
|||
this.module = "Compression";
|
||||
this.description = "Decompresses data using the Bzip2 algorithm.";
|
||||
this.infoURL = "https://wikipedia.org/wiki/Bzip2";
|
||||
this.inputType = "byteArray";
|
||||
this.outputType = "string";
|
||||
this.args = [];
|
||||
this.inputType = "ArrayBuffer";
|
||||
this.outputType = "ArrayBuffer";
|
||||
this.args = [
|
||||
{
|
||||
name: "Use low-memory, slower decompression algorithm",
|
||||
type: "boolean",
|
||||
value: false
|
||||
}
|
||||
];
|
||||
this.patterns = [
|
||||
{
|
||||
"match": "^\\x42\\x5a\\x68",
|
||||
|
|
@ -41,14 +47,24 @@ class Bzip2Decompress extends Operation {
|
|||
* @returns {string}
|
||||
*/
|
||||
run(input, args) {
|
||||
const compressed = new Uint8Array(input);
|
||||
|
||||
try {
|
||||
const bzip2Reader = bzip2.array(compressed);
|
||||
return bzip2.simple(bzip2Reader);
|
||||
} catch (err) {
|
||||
throw new OperationError(err);
|
||||
const [small] = args;
|
||||
if (input.byteLength <= 0) {
|
||||
throw new OperationError("Please provide an input.");
|
||||
}
|
||||
if (ENVIRONMENT_IS_WORKER()) self.sendStatusMessage("Loading Bzip2...");
|
||||
return new Promise((resolve, reject) => {
|
||||
Bzip2().then(bzip2 => {
|
||||
if (ENVIRONMENT_IS_WORKER()) self.sendStatusMessage("Decompressing data...");
|
||||
const inpArray = new Uint8Array(input);
|
||||
const bzip2cc = bzip2.decompressBZ2(inpArray, small ? 1 : 0);
|
||||
if (bzip2cc.error !== 0) {
|
||||
reject(new OperationError(bzip2cc.error_msg));
|
||||
} else {
|
||||
const output = bzip2cc.output;
|
||||
resolve(output.buffer.slice(output.byteOffset, output.byteLength + output.byteOffset));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,8 +4,13 @@
|
|||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import * as d3temp from "d3";
|
||||
import * as nodomtemp from "nodom";
|
||||
|
||||
import Operation from "../Operation";
|
||||
import Utils from "../Utils";
|
||||
|
||||
const d3 = d3temp.default ? d3temp.default : d3temp;
|
||||
const nodom = nodomtemp.default ? nodomtemp.default: nodomtemp;
|
||||
|
||||
/**
|
||||
* Entropy operation
|
||||
|
|
@ -19,30 +24,45 @@ class Entropy extends Operation {
|
|||
super();
|
||||
|
||||
this.name = "Entropy";
|
||||
this.module = "Default";
|
||||
this.module = "Charts";
|
||||
this.description = "Shannon Entropy, in the context of information theory, is a measure of the rate at which information is produced by a source of data. It can be used, in a broad sense, to detect whether data is likely to be structured or unstructured. 8 is the maximum, representing highly unstructured, 'random' data. English language text usually falls somewhere between 3.5 and 5. Properly encrypted or compressed data should have an entropy of over 7.5.";
|
||||
this.infoURL = "https://wikipedia.org/wiki/Entropy_(information_theory)";
|
||||
this.inputType = "byteArray";
|
||||
this.outputType = "number";
|
||||
this.inputType = "ArrayBuffer";
|
||||
this.outputType = "json";
|
||||
this.presentType = "html";
|
||||
this.args = [];
|
||||
this.args = [
|
||||
{
|
||||
"name": "Visualisation",
|
||||
"type": "option",
|
||||
"value": ["Shannon scale", "Histogram (Bar)", "Histogram (Line)", "Curve", "Image"]
|
||||
}
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {byteArray} input
|
||||
* @param {Object[]} args
|
||||
* Calculates the frequency of bytes in the input.
|
||||
*
|
||||
* @param {Uint8Array} input
|
||||
* @returns {number}
|
||||
*/
|
||||
run(input, args) {
|
||||
calculateShannonEntropy(input) {
|
||||
const prob = [],
|
||||
uniques = input.unique(),
|
||||
str = Utils.byteArrayToChars(input);
|
||||
let i;
|
||||
occurrences = new Array(256).fill(0);
|
||||
|
||||
for (i = 0; i < uniques.length; i++) {
|
||||
prob.push(str.count(Utils.chr(uniques[i])) / input.length);
|
||||
// Count occurrences of each byte in the input
|
||||
let i;
|
||||
for (i = 0; i < input.length; i++) {
|
||||
occurrences[input[i]]++;
|
||||
}
|
||||
|
||||
// Store probability list
|
||||
for (i = 0; i < occurrences.length; i++) {
|
||||
if (occurrences[i] > 0) {
|
||||
prob.push(occurrences[i] / input.length);
|
||||
}
|
||||
}
|
||||
|
||||
// Calculate Shannon entropy
|
||||
let entropy = 0,
|
||||
p;
|
||||
|
||||
|
|
@ -54,44 +74,357 @@ class Entropy extends Operation {
|
|||
return -entropy;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates the scanning entropy of the input
|
||||
*
|
||||
* @param {Uint8Array} inputBytes
|
||||
* @returns {Object}
|
||||
*/
|
||||
calculateScanningEntropy(inputBytes) {
|
||||
const entropyData = [];
|
||||
const binWidth = inputBytes.length < 256 ? 8 : 256;
|
||||
|
||||
for (let bytePos = 0; bytePos < inputBytes.length; bytePos += binWidth) {
|
||||
const block = inputBytes.slice(bytePos, bytePos+binWidth);
|
||||
entropyData.push(this.calculateShannonEntropy(block));
|
||||
}
|
||||
|
||||
return { entropyData, binWidth };
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates the frequency of bytes in the input.
|
||||
*
|
||||
* @param {object} svg
|
||||
* @param {function} xScale
|
||||
* @param {function} yScale
|
||||
* @param {integer} svgHeight
|
||||
* @param {integer} svgWidth
|
||||
* @param {object} margins
|
||||
* @param {string} xTitle
|
||||
* @param {string} yTitle
|
||||
*/
|
||||
createAxes(svg, xScale, yScale, svgHeight, svgWidth, margins, title, xTitle, yTitle) {
|
||||
// Axes
|
||||
const yAxis = d3.axisLeft()
|
||||
.scale(yScale);
|
||||
|
||||
const xAxis = d3.axisBottom()
|
||||
.scale(xScale);
|
||||
|
||||
svg.append("g")
|
||||
.attr("transform", `translate(0, ${svgHeight - margins.bottom})`)
|
||||
.call(xAxis);
|
||||
|
||||
svg.append("g")
|
||||
.attr("transform", `translate(${margins.left},0)`)
|
||||
.call(yAxis);
|
||||
|
||||
// Axes labels
|
||||
svg.append("text")
|
||||
.attr("transform", "rotate(-90)")
|
||||
.attr("y", 0 - margins.left)
|
||||
.attr("x", 0 - (svgHeight / 2))
|
||||
.attr("dy", "1em")
|
||||
.style("text-anchor", "middle")
|
||||
.text(yTitle);
|
||||
|
||||
svg.append("text")
|
||||
.attr("transform", `translate(${svgWidth / 2}, ${svgHeight - margins.bottom + 40})`)
|
||||
.style("text-anchor", "middle")
|
||||
.text(xTitle);
|
||||
|
||||
// Add title
|
||||
svg.append("text")
|
||||
.attr("transform", `translate(${svgWidth / 2}, ${margins.top - 10})`)
|
||||
.style("text-anchor", "middle")
|
||||
.text(title);
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates the frequency of bytes in the input.
|
||||
*
|
||||
* @param {Uint8Array} inputBytes
|
||||
* @returns {number[]}
|
||||
*/
|
||||
calculateByteFrequency(inputBytes) {
|
||||
const freq = new Array(256).fill(0);
|
||||
if (inputBytes.length === 0) return freq;
|
||||
|
||||
// Count occurrences of each byte in the input
|
||||
let i;
|
||||
for (i = 0; i < inputBytes.length; i++) {
|
||||
freq[inputBytes[i]]++;
|
||||
}
|
||||
|
||||
for (i = 0; i < freq.length; i++) {
|
||||
freq[i] = freq[i] / inputBytes.length;
|
||||
}
|
||||
|
||||
return freq;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates the frequency of bytes in the input.
|
||||
*
|
||||
* @param {number[]} byteFrequency
|
||||
* @returns {HTML}
|
||||
*/
|
||||
createByteFrequencyLineHistogram(byteFrequency) {
|
||||
const margins = { top: 30, right: 20, bottom: 50, left: 30 };
|
||||
|
||||
const svgWidth = 500,
|
||||
svgHeight = 500;
|
||||
|
||||
const document = new nodom.Document();
|
||||
let svg = document.createElement("svg");
|
||||
|
||||
svg = d3.select(svg)
|
||||
.attr("width", "100%")
|
||||
.attr("height", "100%")
|
||||
.attr("viewBox", `0 0 ${svgWidth} ${svgHeight}`);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(byteFrequency, d => d)])
|
||||
.range([svgHeight - margins.bottom, margins.top]);
|
||||
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, byteFrequency.length - 1])
|
||||
.range([margins.left, svgWidth - margins.right]);
|
||||
|
||||
const line = d3.line()
|
||||
.x((_, i) => xScale(i))
|
||||
.y(d => yScale(d))
|
||||
.curve(d3.curveMonotoneX);
|
||||
|
||||
svg.append("path")
|
||||
.datum(byteFrequency)
|
||||
.attr("fill", "none")
|
||||
.attr("stroke", "steelblue")
|
||||
.attr("d", line);
|
||||
|
||||
this.createAxes(svg, xScale, yScale, svgHeight, svgWidth, margins, "", "Byte", "Byte Frequency");
|
||||
|
||||
return svg._groups[0][0].outerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a byte frequency histogram
|
||||
*
|
||||
* @param {number[]} byteFrequency
|
||||
* @returns {HTML}
|
||||
*/
|
||||
createByteFrequencyBarHistogram(byteFrequency) {
|
||||
const margins = { top: 30, right: 20, bottom: 50, left: 30 };
|
||||
|
||||
const svgWidth = 500,
|
||||
svgHeight = 500,
|
||||
binWidth = 1;
|
||||
|
||||
const document = new nodom.Document();
|
||||
let svg = document.createElement("svg");
|
||||
svg = d3.select(svg)
|
||||
.attr("width", "100%")
|
||||
.attr("height", "100%")
|
||||
.attr("viewBox", `0 0 ${svgWidth} ${svgHeight}`);
|
||||
|
||||
const yExtent = d3.extent(byteFrequency, d => d);
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain(yExtent)
|
||||
.range([svgHeight - margins.bottom, margins.top]);
|
||||
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, byteFrequency.length - 1])
|
||||
.range([margins.left - binWidth, svgWidth - margins.right]);
|
||||
|
||||
svg.selectAll("rect")
|
||||
.data(byteFrequency)
|
||||
.enter().append("rect")
|
||||
.attr("x", (_, i) => xScale(i) + binWidth)
|
||||
.attr("y", dataPoint => yScale(dataPoint))
|
||||
.attr("width", binWidth)
|
||||
.attr("height", dataPoint => yScale(yExtent[0]) - yScale(dataPoint))
|
||||
.attr("fill", "blue");
|
||||
|
||||
this.createAxes(svg, xScale, yScale, svgHeight, svgWidth, margins, "", "Byte", "Byte Frequency");
|
||||
|
||||
return svg._groups[0][0].outerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a byte frequency histogram
|
||||
*
|
||||
* @param {number[]} entropyData
|
||||
* @returns {HTML}
|
||||
*/
|
||||
createEntropyCurve(entropyData) {
|
||||
const margins = { top: 30, right: 20, bottom: 50, left: 30 };
|
||||
|
||||
const svgWidth = 500,
|
||||
svgHeight = 500;
|
||||
|
||||
const document = new nodom.Document();
|
||||
let svg = document.createElement("svg");
|
||||
svg = d3.select(svg)
|
||||
.attr("width", "100%")
|
||||
.attr("height", "100%")
|
||||
.attr("viewBox", `0 0 ${svgWidth} ${svgHeight}`);
|
||||
|
||||
const yScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(entropyData, d => d)])
|
||||
.range([svgHeight - margins.bottom, margins.top]);
|
||||
|
||||
const xScale = d3.scaleLinear()
|
||||
.domain([0, entropyData.length])
|
||||
.range([margins.left, svgWidth - margins.right]);
|
||||
|
||||
const line = d3.line()
|
||||
.x((_, i) => xScale(i))
|
||||
.y(d => yScale(d))
|
||||
.curve(d3.curveMonotoneX);
|
||||
|
||||
if (entropyData.length > 0) {
|
||||
svg.append("path")
|
||||
.datum(entropyData)
|
||||
.attr("d", line);
|
||||
|
||||
svg.selectAll("path").attr("fill", "none").attr("stroke", "steelblue");
|
||||
}
|
||||
|
||||
this.createAxes(svg, xScale, yScale, svgHeight, svgWidth, margins, "Scanning Entropy", "Block", "Entropy");
|
||||
|
||||
return svg._groups[0][0].outerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates an image representation of the entropy
|
||||
*
|
||||
* @param {number[]} entropyData
|
||||
* @returns {HTML}
|
||||
*/
|
||||
createEntropyImage(entropyData) {
|
||||
const svgHeight = 100,
|
||||
svgWidth = 100,
|
||||
cellSize = 1,
|
||||
nodes = [];
|
||||
|
||||
for (let i = 0; i < entropyData.length; i++) {
|
||||
nodes.push({
|
||||
x: i % svgWidth,
|
||||
y: Math.floor(i / svgWidth),
|
||||
entropy: entropyData[i]
|
||||
});
|
||||
}
|
||||
|
||||
const document = new nodom.Document();
|
||||
let svg = document.createElement("svg");
|
||||
svg = d3.select(svg)
|
||||
.attr("width", "100%")
|
||||
.attr("height", "100%")
|
||||
.attr("viewBox", `0 0 ${svgWidth} ${svgHeight}`);
|
||||
|
||||
const greyScale = d3.scaleLinear()
|
||||
.domain([0, d3.max(entropyData, d => d)])
|
||||
.range(["#000000", "#FFFFFF"])
|
||||
.interpolate(d3.interpolateRgb);
|
||||
|
||||
svg
|
||||
.selectAll("rect")
|
||||
.data(nodes)
|
||||
.enter().append("rect")
|
||||
.attr("x", d => d.x * cellSize)
|
||||
.attr("y", d => d.y * cellSize)
|
||||
.attr("width", cellSize)
|
||||
.attr("height", cellSize)
|
||||
.style("fill", d => greyScale(d.entropy));
|
||||
|
||||
return svg._groups[0][0].outerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Displays the entropy as a scale bar for web apps.
|
||||
*
|
||||
* @param {number} entropy
|
||||
* @returns {html}
|
||||
* @returns {HTML}
|
||||
*/
|
||||
present(entropy) {
|
||||
createShannonEntropyVisualization(entropy) {
|
||||
return `Shannon entropy: ${entropy}
|
||||
<br><canvas id='chart-area'></canvas><br>
|
||||
- 0 represents no randomness (i.e. all the bytes in the data have the same value) whereas 8, the maximum, represents a completely random string.
|
||||
- Standard English text usually falls somewhere between 3.5 and 5.
|
||||
- Properly encrypted or compressed data of a reasonable length should have an entropy of over 7.5.
|
||||
<br><canvas id='chart-area'></canvas><br>
|
||||
- 0 represents no randomness (i.e. all the bytes in the data have the same value) whereas 8, the maximum, represents a completely random string.
|
||||
- Standard English text usually falls somewhere between 3.5 and 5.
|
||||
- Properly encrypted or compressed data of a reasonable length should have an entropy of over 7.5.
|
||||
|
||||
The following results show the entropy of chunks of the input data. Chunks with particularly high entropy could suggest encrypted or compressed sections.
|
||||
The following results show the entropy of chunks of the input data. Chunks with particularly high entropy could suggest encrypted or compressed sections.
|
||||
|
||||
<br><script>
|
||||
var canvas = document.getElementById("chart-area"),
|
||||
parentRect = canvas.parentNode.getBoundingClientRect(),
|
||||
entropy = ${entropy},
|
||||
height = parentRect.height * 0.25;
|
||||
<br><script>
|
||||
var canvas = document.getElementById("chart-area"),
|
||||
parentRect = canvas.parentNode.getBoundingClientRect(),
|
||||
entropy = ${entropy},
|
||||
height = parentRect.height * 0.25;
|
||||
|
||||
canvas.width = parentRect.width * 0.95;
|
||||
canvas.height = height > 150 ? 150 : height;
|
||||
canvas.width = parentRect.width * 0.95;
|
||||
canvas.height = height > 150 ? 150 : height;
|
||||
|
||||
CanvasComponents.drawScaleBar(canvas, entropy, 8, [
|
||||
{
|
||||
label: "English text",
|
||||
min: 3.5,
|
||||
max: 5
|
||||
},{
|
||||
label: "Encrypted/compressed",
|
||||
min: 7.5,
|
||||
max: 8
|
||||
}
|
||||
]);
|
||||
</script>`;
|
||||
CanvasComponents.drawScaleBar(canvas, entropy, 8, [
|
||||
{
|
||||
label: "English text",
|
||||
min: 3.5,
|
||||
max: 5
|
||||
},{
|
||||
label: "Encrypted/compressed",
|
||||
min: 7.5,
|
||||
max: 8
|
||||
}
|
||||
]);
|
||||
</script>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {ArrayBuffer} input
|
||||
* @param {Object[]} args
|
||||
* @returns {json}
|
||||
*/
|
||||
run(input, args) {
|
||||
const visualizationType = args[0];
|
||||
input = new Uint8Array(input);
|
||||
|
||||
switch (visualizationType) {
|
||||
case "Histogram (Bar)":
|
||||
case "Histogram (Line)":
|
||||
return this.calculateByteFrequency(input);
|
||||
case "Curve":
|
||||
case "Image":
|
||||
return this.calculateScanningEntropy(input).entropyData;
|
||||
case "Shannon scale":
|
||||
default:
|
||||
return this.calculateShannonEntropy(input);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Displays the entropy in a visualisation for web apps.
|
||||
*
|
||||
* @param {json} entropyData
|
||||
* @param {Object[]} args
|
||||
* @returns {html}
|
||||
*/
|
||||
present(entropyData, args) {
|
||||
const visualizationType = args[0];
|
||||
|
||||
switch (visualizationType) {
|
||||
case "Histogram (Bar)":
|
||||
return this.createByteFrequencyBarHistogram(entropyData);
|
||||
case "Histogram (Line)":
|
||||
return this.createByteFrequencyLineHistogram(entropyData);
|
||||
case "Curve":
|
||||
return this.createEntropyCurve(entropyData);
|
||||
case "Image":
|
||||
return this.createEntropyImage(entropyData);
|
||||
case "Shannon scale":
|
||||
default:
|
||||
return this.createShannonEntropyVisualization(entropyData);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export default Entropy;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,107 @@
|
|||
/**
|
||||
* @author George O [georgeomnet+cyberchef@gmail.com]
|
||||
* @copyright Crown Copyright 2019
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
|
||||
import Operation from "../Operation";
|
||||
import Utils from "../Utils";
|
||||
|
||||
/**
|
||||
* Index of Coincidence operation
|
||||
*/
|
||||
class IndexOfCoincidence extends Operation {
|
||||
|
||||
/**
|
||||
* IndexOfCoincidence constructor
|
||||
*/
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
this.name = "Index of Coincidence";
|
||||
this.module = "Default";
|
||||
this.description = "Index of Coincidence (IC) is the probability of two randomly selected characters being the same. This can be used to determine whether text is readable or random, with English text having an IC of around 0.066. IC can therefore be a sound method to automate frequency analysis.";
|
||||
this.infoURL = "https://wikipedia.org/wiki/Index_of_coincidence";
|
||||
this.inputType = "string";
|
||||
this.outputType = "number";
|
||||
this.presentType = "html";
|
||||
this.args = [];
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} input
|
||||
* @param {Object[]} args
|
||||
* @returns {number}
|
||||
*/
|
||||
run(input, args) {
|
||||
const text = input.toLowerCase().replace(/[^a-z]/g, ""),
|
||||
frequencies = new Array(26).fill(0),
|
||||
alphabet = Utils.expandAlphRange("a-z");
|
||||
let coincidence = 0.00,
|
||||
density = 0.00,
|
||||
result = 0.00,
|
||||
i;
|
||||
|
||||
for (i=0; i < alphabet.length; i++) {
|
||||
frequencies[i] = text.count(alphabet[i]);
|
||||
}
|
||||
|
||||
for (i=0; i < frequencies.length; i++) {
|
||||
coincidence += frequencies[i] * (frequencies[i] - 1);
|
||||
}
|
||||
|
||||
density = frequencies.sum();
|
||||
|
||||
// Ensure that we don't divide by 0
|
||||
if (density < 2) density = 2;
|
||||
|
||||
result = coincidence / (density * (density - 1));
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Displays the IC as a scale bar for web apps.
|
||||
*
|
||||
* @param {number} ic
|
||||
* @returns {html}
|
||||
*/
|
||||
present(ic) {
|
||||
return `Index of Coincidence: ${ic}
|
||||
Normalized: ${ic * 26}
|
||||
<br><canvas id='chart-area'></canvas><br>
|
||||
- 0 represents complete randomness (all characters are unique), whereas 1 represents no randomness (all characters are identical).
|
||||
- English text generally has an IC of between 0.67 to 0.78.
|
||||
- 'Random' text is determined by the probability that each letter occurs the same number of times as another.
|
||||
|
||||
The graph shows the IC of the input data. A low IC generally means that the text is random, compressed or encrypted.
|
||||
|
||||
<script type='application/javascript'>
|
||||
var canvas = document.getElementById("chart-area"),
|
||||
parentRect = canvas.parentNode.getBoundingClientRect(),
|
||||
ic = ${ic};
|
||||
|
||||
canvas.width = parentRect.width * 0.95;
|
||||
canvas.height = parentRect.height * 0.25;
|
||||
|
||||
ic = ic > 0.25 ? 0.25 : ic;
|
||||
|
||||
CanvasComponents.drawScaleBar(canvas, ic, 0.25, [
|
||||
{
|
||||
label: "English text",
|
||||
min: 0.05,
|
||||
max: 0.08
|
||||
},
|
||||
{
|
||||
label: "> 0.25",
|
||||
min: 0.24,
|
||||
max: 0.25
|
||||
}
|
||||
]);
|
||||
</script>
|
||||
`;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
export default IndexOfCoincidence;
|
||||
|
|
@ -51,6 +51,10 @@ class JSONToCSV extends Operation {
|
|||
this.rowDelim = rowDelim;
|
||||
const self = this;
|
||||
|
||||
if (!(input instanceof Array)) {
|
||||
input = [input];
|
||||
}
|
||||
|
||||
try {
|
||||
// If the JSON is an array of arrays, this is easy
|
||||
if (input[0] instanceof Array) {
|
||||
|
|
@ -89,6 +93,8 @@ class JSONToCSV extends Operation {
|
|||
* @returns {string}
|
||||
*/
|
||||
escapeCellContents(data) {
|
||||
if (typeof data === "number") data = data.toString();
|
||||
|
||||
// Double quotes should be doubled up
|
||||
data = data.replace(/"/g, '""');
|
||||
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ class UnescapeString extends Operation {
|
|||
|
||||
this.name = "Unescape string";
|
||||
this.module = "Default";
|
||||
this.description = "Unescapes characters in a string that have been escaped. For example, <code>Don\\'t stop me now</code> becomes <code>Don't stop me now</code>.<br><br>Supports the following escape sequences:<ul><li><code>\\n</code> (Line feed/newline)</li><li><code>\\r</code> (Carriage return)</li><li><code>\\t</code> (Horizontal tab)</li><li><code>\\b</code> (Backspace)</li><li><code>\\f</code> (Form feed)</li><li><code>\\xnn</code> (Hex, where n is 0-f)</li><li><code>\\\\</code> (Backslash)</li><li><code>\\'</code> (Single quote)</li><li><code>\\"</code> (Double quote)</li><li><code>\\unnnn</code> (Unicode character)</li><li><code>\\u{nnnnnn}</code> (Unicode code point)</li></ul>";
|
||||
this.description = "Unescapes characters in a string that have been escaped. For example, <code>Don\\'t stop me now</code> becomes <code>Don't stop me now</code>.<br><br>Supports the following escape sequences:<ul><li><code>\\n</code> (Line feed/newline)</li><li><code>\\r</code> (Carriage return)</li><li><code>\\t</code> (Horizontal tab)</li><li><code>\\b</code> (Backspace)</li><li><code>\\f</code> (Form feed)</li><li><code>\\nnn</code> (Octal, where n is 0-7)</li><li><code>\\xnn</code> (Hex, where n is 0-f)</li><li><code>\\\\</code> (Backslash)</li><li><code>\\'</code> (Single quote)</li><li><code>\\"</code> (Double quote)</li><li><code>\\unnnn</code> (Unicode character)</li><li><code>\\u{nnnnnn}</code> (Unicode code point)</li></ul>";
|
||||
this.infoURL = "https://wikipedia.org/wiki/Escape_sequence";
|
||||
this.inputType = "string";
|
||||
this.outputType = "string";
|
||||
|
|
|
|||
|
|
@ -1,265 +0,0 @@
|
|||
/** @license
|
||||
========================================================================
|
||||
bzip2.js - a small bzip2 decompression implementation
|
||||
|
||||
Copyright 2011 by antimatter15 (antimatter15@gmail.com)
|
||||
|
||||
Based on micro-bunzip by Rob Landley (rob@landley.net).
|
||||
|
||||
Copyright (c) 2011 by antimatter15 (antimatter15@gmail.com).
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a
|
||||
copy of this software and associated documentation files (the "Software"),
|
||||
to deal in the Software without restriction, including without limitation
|
||||
the rights to use, copy, modify, merge, publish, distribute, sublicense,
|
||||
and/or sell copies of the Software, and to permit persons to whom the
|
||||
Software is furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included
|
||||
in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
||||
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
|
||||
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
||||
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH
|
||||
THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
"use strict";
|
||||
|
||||
var bzip2 = {};
|
||||
|
||||
bzip2.array = function(bytes){
|
||||
var bit = 0, byte = 0;
|
||||
var BITMASK = [0, 0x01, 0x03, 0x07, 0x0F, 0x1F, 0x3F, 0x7F, 0xFF ];
|
||||
return function(n){
|
||||
var result = 0;
|
||||
while(n > 0){
|
||||
var left = 8 - bit;
|
||||
if(n >= left){
|
||||
result <<= left;
|
||||
result |= (BITMASK[left] & bytes[byte++]);
|
||||
bit = 0;
|
||||
n -= left;
|
||||
}else{
|
||||
result <<= n;
|
||||
result |= ((bytes[byte] & (BITMASK[n] << (8 - n - bit))) >> (8 - n - bit));
|
||||
bit += n;
|
||||
n = 0;
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
}
|
||||
|
||||
bzip2.simple = function(bits){
|
||||
var size = bzip2.header(bits);
|
||||
var all = '', chunk = '';
|
||||
do{
|
||||
all += chunk;
|
||||
chunk = bzip2.decompress(bits, size);
|
||||
}while(chunk != -1);
|
||||
return all;
|
||||
}
|
||||
|
||||
bzip2.header = function(bits){
|
||||
if(bits(8*3) != 4348520) throw "No magic number found";
|
||||
var i = bits(8) - 48;
|
||||
if(i < 1 || i > 9) throw "Not a BZIP archive";
|
||||
return i;
|
||||
};
|
||||
|
||||
|
||||
//takes a function for reading the block data (starting with 0x314159265359)
|
||||
//a block size (0-9) (optional, defaults to 9)
|
||||
//a length at which to stop decompressing and return the output
|
||||
bzip2.decompress = function(bits, size, len){
|
||||
var MAX_HUFCODE_BITS = 20;
|
||||
var MAX_SYMBOLS = 258;
|
||||
var SYMBOL_RUNA = 0;
|
||||
var SYMBOL_RUNB = 1;
|
||||
var GROUP_SIZE = 50;
|
||||
|
||||
var bufsize = 100000 * size;
|
||||
for(var h = '', i = 0; i < 6; i++) h += bits(8).toString(16);
|
||||
if(h == "177245385090") return -1; //last block
|
||||
if(h != "314159265359") throw "Not valid bzip data";
|
||||
bits(32); //ignore CRC codes
|
||||
if(bits(1)) throw "Unsupported obsolete version";
|
||||
var origPtr = bits(24);
|
||||
if(origPtr > bufsize) throw "Initial position larger than buffer size";
|
||||
var t = bits(16);
|
||||
var symToByte = new Uint8Array(256),
|
||||
symTotal = 0;
|
||||
for (i = 0; i < 16; i++) {
|
||||
if(t & (1 << (15 - i))) {
|
||||
var k = bits(16);
|
||||
for(j = 0; j < 16; j++){
|
||||
if(k & (1 << (15 - j))){
|
||||
symToByte[symTotal++] = (16 * i) + j;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var groupCount = bits(3);
|
||||
if(groupCount < 2 || groupCount > 6) throw "Error 1";
|
||||
var nSelectors = bits(15);
|
||||
if(nSelectors == 0) throw "Error";
|
||||
var mtfSymbol = []; //TODO: possibly replace JS array with typed arrays
|
||||
for(var i = 0; i < groupCount; i++) mtfSymbol[i] = i;
|
||||
var selectors = new Uint8Array(32768);
|
||||
|
||||
for(var i = 0; i < nSelectors; i++){
|
||||
for(var j = 0; bits(1); j++) if(j >= groupCount) throw "Error 2";
|
||||
var uc = mtfSymbol[j];
|
||||
mtfSymbol.splice(j, 1); //this is a probably inefficient MTF transform
|
||||
mtfSymbol.splice(0, 0, uc);
|
||||
selectors[i] = uc;
|
||||
}
|
||||
|
||||
var symCount = symTotal + 2;
|
||||
var groups = [];
|
||||
for(var j = 0; j < groupCount; j++){
|
||||
var length = new Uint8Array(MAX_SYMBOLS),
|
||||
temp = new Uint8Array(MAX_HUFCODE_BITS+1);
|
||||
t = bits(5); //lengths
|
||||
for(var i = 0; i < symCount; i++){
|
||||
while(true){
|
||||
if (t < 1 || t > MAX_HUFCODE_BITS) throw "Error 3";
|
||||
if(!bits(1)) break;
|
||||
if(!bits(1)) t++;
|
||||
else t--;
|
||||
}
|
||||
length[i] = t;
|
||||
}
|
||||
var minLen, maxLen;
|
||||
minLen = maxLen = length[0];
|
||||
for(var i = 1; i < symCount; i++){
|
||||
if(length[i] > maxLen) maxLen = length[i];
|
||||
else if(length[i] < minLen) minLen = length[i];
|
||||
}
|
||||
var hufGroup;
|
||||
hufGroup = groups[j] = {};
|
||||
hufGroup.permute = new Uint32Array(MAX_SYMBOLS);
|
||||
hufGroup.limit = new Uint32Array(MAX_HUFCODE_BITS + 1);
|
||||
hufGroup.base = new Uint32Array(MAX_HUFCODE_BITS + 1);
|
||||
hufGroup.minLen = minLen;
|
||||
hufGroup.maxLen = maxLen;
|
||||
var base = hufGroup.base.subarray(1);
|
||||
var limit = hufGroup.limit.subarray(1);
|
||||
var pp = 0;
|
||||
for(var i = minLen; i <= maxLen; i++)
|
||||
for(var t = 0; t < symCount; t++)
|
||||
if(length[t] == i) hufGroup.permute[pp++] = t;
|
||||
for(i = minLen; i <= maxLen; i++) temp[i] = limit[i] = 0;
|
||||
for(i = 0; i < symCount; i++) temp[length[i]]++;
|
||||
pp = t = 0;
|
||||
for(i = minLen; i < maxLen; i++) {
|
||||
pp += temp[i];
|
||||
limit[i] = pp - 1;
|
||||
pp <<= 1;
|
||||
base[i+1] = pp - (t += temp[i]);
|
||||
}
|
||||
limit[maxLen]=pp+temp[maxLen]-1;
|
||||
base[minLen]=0;
|
||||
}
|
||||
var byteCount = new Uint32Array(256);
|
||||
for(var i = 0; i < 256; i++) mtfSymbol[i] = i;
|
||||
var runPos, count, symCount, selector;
|
||||
runPos = count = symCount = selector = 0;
|
||||
var buf = new Uint32Array(bufsize);
|
||||
while(true){
|
||||
if(!(symCount--)){
|
||||
symCount = GROUP_SIZE - 1;
|
||||
if(selector >= nSelectors) throw "Error 4";
|
||||
hufGroup = groups[selectors[selector++]];
|
||||
base = hufGroup.base.subarray(1);
|
||||
limit = hufGroup.limit.subarray(1);
|
||||
}
|
||||
i = hufGroup.minLen;
|
||||
j = bits(i);
|
||||
while(true){
|
||||
if(i > hufGroup.maxLen) throw "Error 5";
|
||||
if(j <= limit[i]) break;
|
||||
i++;
|
||||
j = (j << 1) | bits(1);
|
||||
}
|
||||
j -= base[i];
|
||||
if(j < 0 || j >= MAX_SYMBOLS) throw "Error 6";
|
||||
var nextSym = hufGroup.permute[j];
|
||||
if (nextSym == SYMBOL_RUNA || nextSym == SYMBOL_RUNB) {
|
||||
if(!runPos){
|
||||
runPos = 1;
|
||||
t = 0;
|
||||
}
|
||||
if(nextSym == SYMBOL_RUNA) t += runPos;
|
||||
else t += 2 * runPos;
|
||||
runPos <<= 1;
|
||||
continue;
|
||||
}
|
||||
if(runPos){
|
||||
runPos = 0;
|
||||
if(count + t >= bufsize) throw "Error 7";
|
||||
uc = symToByte[mtfSymbol[0]];
|
||||
byteCount[uc] += t;
|
||||
while(t--) buf[count++] = uc;
|
||||
}
|
||||
if(nextSym > symTotal) break;
|
||||
if(count >= bufsize) throw "Error 8";
|
||||
i = nextSym -1;
|
||||
uc = mtfSymbol[i];
|
||||
mtfSymbol.splice(i, 1);
|
||||
mtfSymbol.splice(0, 0, uc);
|
||||
uc = symToByte[uc];
|
||||
byteCount[uc]++;
|
||||
buf[count++] = uc;
|
||||
}
|
||||
if(origPtr < 0 || origPtr >= count) throw "Error 9";
|
||||
var j = 0;
|
||||
for(var i = 0; i < 256; i++){
|
||||
k = j + byteCount[i];
|
||||
byteCount[i] = j;
|
||||
j = k;
|
||||
}
|
||||
for(var i = 0; i < count; i++){
|
||||
uc = buf[i] & 0xff;
|
||||
buf[byteCount[uc]] |= (i << 8);
|
||||
byteCount[uc]++;
|
||||
}
|
||||
var pos = 0, current = 0, run = 0;
|
||||
if(count) {
|
||||
pos = buf[origPtr];
|
||||
current = (pos & 0xff);
|
||||
pos >>= 8;
|
||||
run = -1;
|
||||
}
|
||||
count = count;
|
||||
var output = '';
|
||||
var copies, previous, outbyte;
|
||||
if(!len) len = Infinity;
|
||||
while(count){
|
||||
count--;
|
||||
previous = current;
|
||||
pos = buf[pos];
|
||||
current = pos & 0xff;
|
||||
pos >>= 8;
|
||||
if(run++ == 3){
|
||||
copies = current;
|
||||
outbyte = previous;
|
||||
current = -1;
|
||||
}else{
|
||||
copies = 1;
|
||||
outbyte = current;
|
||||
}
|
||||
while(copies--){
|
||||
output += (String.fromCharCode(outbyte));
|
||||
if(!--len) return output;
|
||||
}
|
||||
if(current != previous) run = 0;
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
export default bzip2;
|
||||
|
|
@ -264,9 +264,9 @@ class App {
|
|||
|
||||
this.columnSplitter = Split(["#operations", "#recipe", "#IO"], {
|
||||
sizes: [20, 30, 50],
|
||||
minSize: minimise ? [0, 0, 0] : [240, 370, 450],
|
||||
minSize: minimise ? [0, 0, 0] : [240, 310, 450],
|
||||
gutterSize: 4,
|
||||
expandToMin: false,
|
||||
expandToMin: true,
|
||||
onDrag: this.debounce(function() {
|
||||
this.manager.recipe.adjustWidth();
|
||||
this.manager.input.calcMaxTabs();
|
||||
|
|
|
|||
|
|
@ -191,7 +191,7 @@
|
|||
<ul id="rec-list" class="list-area no-select"></ul>
|
||||
|
||||
<div id="controls" class="no-select">
|
||||
<div class="d-flex align-items-center">
|
||||
<div id="controls-content" class="d-flex align-items-center">
|
||||
<button type="button" class="mx-2 btn btn-lg btn-secondary" id="step" data-toggle="tooltip" title="Step through the recipe">
|
||||
Step
|
||||
</button>
|
||||
|
|
|
|||
|
|
@ -122,7 +122,7 @@ div.toggle-string {
|
|||
}
|
||||
|
||||
.operation .form-control {
|
||||
padding: 20px 12px 6px 12px;
|
||||
padding: 20px 12px 6px 12px !important;
|
||||
border-top-left-radius: 4px;
|
||||
border-top-right-radius: 4px;
|
||||
background-image: none;
|
||||
|
|
|
|||
|
|
@ -21,6 +21,14 @@
|
|||
background-color: var(--secondary-background-colour);
|
||||
}
|
||||
|
||||
#controls-content {
|
||||
position: relative;
|
||||
left: 50%;
|
||||
top: 50%;
|
||||
transform: translate(-50%, -50%);
|
||||
transform-origin: center left;
|
||||
}
|
||||
|
||||
#auto-bake-label {
|
||||
display: inline-block;
|
||||
width: 100px;
|
||||
|
|
|
|||
|
|
@ -124,16 +124,21 @@ class RecipeWaiter {
|
|||
* @param {event} evt
|
||||
*/
|
||||
opSortEnd(evt) {
|
||||
if (this.removeIntent) {
|
||||
if (evt.item.parentNode.id === "rec-list") {
|
||||
evt.item.remove();
|
||||
}
|
||||
if (this.removeIntent && evt.item.parentNode.id === "rec-list") {
|
||||
evt.item.remove();
|
||||
return;
|
||||
}
|
||||
|
||||
// Reinitialise the popover on the original element in the ops list because for some reason it
|
||||
// gets destroyed and recreated.
|
||||
this.manager.ops.enableOpsListPopovers(evt.clone);
|
||||
// gets destroyed and recreated. If the clone isn't in the ops list, we use the original item instead.
|
||||
let enableOpsElement;
|
||||
if (evt.clone.parentNode && evt.clone.parentNode.classList.contains("op-list")) {
|
||||
enableOpsElement = evt.clone;
|
||||
} else {
|
||||
enableOpsElement = evt.item;
|
||||
$(evt.item).attr("data-toggle", "popover");
|
||||
}
|
||||
this.manager.ops.enableOpsListPopovers(enableOpsElement);
|
||||
|
||||
if (evt.item.parentNode.id !== "rec-list") {
|
||||
return;
|
||||
|
|
@ -612,6 +617,23 @@ class RecipeWaiter {
|
|||
ingredientRule.style.gridTemplateColumns = "auto auto auto auto";
|
||||
ingredientChildRule.style.gridColumn = "1 / span 4";
|
||||
}
|
||||
|
||||
// Hide Chef icon on Bake button if the page is compressed
|
||||
const bakeIcon = document.querySelector("#bake img");
|
||||
|
||||
if (recList.clientWidth < 370) {
|
||||
// Hide Chef icon on Bake button
|
||||
bakeIcon.style.display = "none";
|
||||
} else {
|
||||
bakeIcon.style.display = "inline-block";
|
||||
}
|
||||
|
||||
// Scale controls to fit pane width
|
||||
const controls = document.getElementById("controls");
|
||||
const controlsContent = document.getElementById("controls-content");
|
||||
const scale = (controls.clientWidth - 1) / controlsContent.scrollWidth;
|
||||
|
||||
controlsContent.style.transform = `translate(-50%, -50%) scale(${scale})`;
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -49,9 +49,11 @@ import "./tests/Hash";
|
|||
import "./tests/HaversineDistance";
|
||||
import "./tests/Hexdump";
|
||||
import "./tests/Image";
|
||||
import "./tests/IndexOfCoincidence";
|
||||
import "./tests/Jump";
|
||||
import "./tests/JSONBeautify";
|
||||
import "./tests/JSONMinify";
|
||||
import "./tests/JSONtoCSV";
|
||||
import "./tests/JWTDecode";
|
||||
import "./tests/JWTSign";
|
||||
import "./tests/JWTVerify";
|
||||
|
|
|
|||
|
|
@ -0,0 +1,22 @@
|
|||
/**
|
||||
* Index of Coincidence tests.
|
||||
*
|
||||
* @author George O [georgeomnet+cyberchef@gmail.com]
|
||||
* @copyright Crown Copyright 2019
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
import TestRegister from "../TestRegister";
|
||||
|
||||
TestRegister.addTests([
|
||||
{
|
||||
name: "Index of Coincidence",
|
||||
input: "Hello world, this is a test to determine the correct IC value.",
|
||||
expectedMatch: /^Index of Coincidence: 0\.07142857142857142\nNormalized: 1\.857142857142857/,
|
||||
recipeConfig: [
|
||||
{
|
||||
"op": "Index of Coincidence",
|
||||
"args": []
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
|
|
@ -0,0 +1,93 @@
|
|||
/**
|
||||
* JSON to CSV tests.
|
||||
*
|
||||
* @author mshwed [m@ttshwed.com]
|
||||
*
|
||||
* @copyright Crown Copyright 2019
|
||||
* @license Apache-2.0
|
||||
*/
|
||||
import TestRegister from "../TestRegister";
|
||||
|
||||
const EXPECTED_CSV_SINGLE = "a,b,c\r\n1,2,3\r\n";
|
||||
const EXPECTED_CSV_MULTIPLE = "a,b,c\r\n1,2,3\r\n1,2,3\r\n";
|
||||
const EXPECTED_CSV_EMPTY = "\r\n\r\n";
|
||||
|
||||
TestRegister.addTests([
|
||||
{
|
||||
name: "JSON to CSV: strings as values",
|
||||
input: JSON.stringify({a: "1", b: "2", c: "3"}),
|
||||
expectedOutput: EXPECTED_CSV_SINGLE,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "JSON to CSV: numbers as values",
|
||||
input: JSON.stringify({a: 1, b: 2, c: 3}),
|
||||
expectedOutput: EXPECTED_CSV_SINGLE,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "JSON to CSV: numbers and strings as values",
|
||||
input: JSON.stringify({a: 1, b: "2", c: 3}),
|
||||
expectedOutput: EXPECTED_CSV_SINGLE,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "JSON to CSV: JSON as an array",
|
||||
input: JSON.stringify([{a: 1, b: "2", c: 3}]),
|
||||
expectedOutput: EXPECTED_CSV_SINGLE,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "JSON to CSV: multiple JSON values in an array",
|
||||
input: JSON.stringify([{a: 1, b: "2", c: 3}, {a: 1, b: "2", c: 3}]),
|
||||
expectedOutput: EXPECTED_CSV_MULTIPLE,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "JSON to CSV: empty JSON",
|
||||
input: JSON.stringify({}),
|
||||
expectedOutput: EXPECTED_CSV_EMPTY,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "JSON to CSV: empty JSON in array",
|
||||
input: JSON.stringify([{}]),
|
||||
expectedOutput: EXPECTED_CSV_EMPTY,
|
||||
recipeConfig: [
|
||||
{
|
||||
op: "JSON to CSV",
|
||||
args: [",", "\\r\\n"]
|
||||
},
|
||||
],
|
||||
}
|
||||
]);
|
||||
Loading…
Reference in New Issue