164e728ad3
Merge pull request #197 from snshn/addetional-black-box-test-data
...
Additional black box test data
2020-07-06 16:51:49 -04:00
8883bd6aca
add more black box test data
2020-07-06 16:15:57 -04:00
ec85121d28
update help message
2020-07-01 06:29:56 -04:00
decd5b2119
add ASCII logo atop of help message
2020-07-01 06:13:58 -04:00
23de5ced21
add tests for utils::indent()
2020-06-28 16:15:42 -04:00
bc98aca2a2
indent items in retrieval log to form a tree-like structure
2020-06-28 16:11:15 -04:00
438ebd520a
Merge pull request #193 from snshn/options-struct
...
Pass options object instead of using separate parameters
2020-06-28 01:51:05 -04:00
ddb97009e9
pass options object instead of using separate parameters
2020-06-28 01:36:41 -04:00
9e5d8ec691
add more sample data for blackbox tests
2020-06-27 14:55:10 -04:00
29bf042da0
trim CSS if it contains nothing but whitespaces
2020-06-26 23:26:55 -04:00
2ac964fae5
include font-src into CSP
2020-06-26 18:14:46 -04:00
ae5d6d2df4
refactor CSP code
2020-06-26 16:19:44 -04:00
44cac65a83
automatically remove "Refresh" and "Location" META tags
2020-06-26 01:18:52 -04:00
0347fd3985
move metadata tag code into a function
2020-06-25 18:23:56 -04:00
5c229c51da
move functions related to URL manipulation into url.rs
2020-06-24 03:16:40 -04:00
f6ea16b3ad
create a separate function for appending URL fragments
2020-06-24 02:26:05 -04:00
f9aac6f41b
update crates
2020-06-20 01:05:39 -04:00
4e31d0433e
only attempt to remove credentals from HTTP(S) URLs
2020-06-01 05:28:02 -04:00
090d647390
group all tests into either passing or failing groups
2020-05-23 03:49:04 -04:00
cbe3f9f554
implement support for embedding images within srcset
2020-05-17 14:06:44 -04:00
a18df74946
refactor code and implement integrity validation
2020-05-12 02:51:37 -04:00
c4569343a4
improve metadata comments
2020-04-30 20:23:09 -04:00
5f5820c71a
Merge pull request #168 from snshn/context-comment
...
Metadata comment tag
2020-04-30 20:06:40 -04:00
c999359b9f
Merge branch 'context-comment' of github.com:Alch-Emi/monolith into context-comment
2020-04-30 19:54:13 -04:00
f22e2b6e68
embed SVG IMAGE assets
2020-04-30 19:51:30 -04:00
9b40dbbf27
add option to exclude web fonts
2020-04-22 09:11:20 -04:00
edacd09dc8
store blobs instead of data URLs in cache
2020-04-19 13:26:14 -04:00
eeea617fb1
escape all special chars within #id and .class CSS selectors
2020-04-11 17:50:23 -04:00
9d3df2cdc6
escape colons within CSS idents
2020-04-10 20:59:56 -04:00
3738be2b6d
treat url()'s found in @counter-style rules as images
2020-04-10 07:22:02 -04:00
594ad55bd8
improve data URL media type detection
2020-04-10 05:50:33 -04:00
c097733ae7
improve parsing of data URLs
2020-04-09 20:27:07 -04:00
349c7bb3ea
properly parse negative units in CSS
2020-04-08 18:07:39 -04:00
a19aa37ea8
Merge pull request #145 from snshn/no-images-svg
...
Empty SVG nodes when excluding images
2020-04-04 15:55:26 -04:00
791e44796e
resolve hrefs of <area> image-map tags
2020-04-04 14:55:45 -04:00
b88479446c
implement unit tests for macros
2020-04-04 08:21:41 -04:00
1d6217ef5a
empty SVG nodes if --no-images
2020-04-03 21:56:46 -04:00
29836d979a
add support for image inputs
2020-04-03 03:30:52 -04:00
643c4ce7ef
implement improvements suggested by @robatipoor
2020-04-03 00:00:08 -04:00
875481b9a2
update help dialog
2020-04-02 03:04:21 -04:00
4951fea730
implement full CSS parsing
2020-04-02 01:09:32 -04:00
be25784297
improve SVG media type detection
2020-03-24 08:50:39 -04:00
479c42e1ce
improve test code structure
2020-03-22 22:08:41 -04:00
933379c798
ensure consistent naming across all tests
2020-03-22 19:03:33 -04:00
59a8be493d
add support for working with local assets
2020-03-22 15:48:23 -04:00
d5ee8ae6ab
account for legacy BODY background="" attribute
2020-03-05 04:56:09 -05:00
7654eec7e2
treat frames the same way as iframes
2020-02-24 20:18:13 -05:00
0d1e21e9ad
add black box tests
2020-02-23 22:48:14 -05:00
3d2d40e7cd
add support for data URL targets
2020-02-23 22:25:37 -05:00
b8b6d8cff6
fix "succeeding" to "passing" in tests
2020-02-23 22:24:33 -05:00
928664dc88
correct is_valid_url to is_http_url
2020-02-23 22:24:33 -05:00
5c8d75539b
rename dataurl to data_url
2020-02-23 22:24:32 -05:00
d9602e25eb
update help dialog and README.md
2020-02-15 01:33:20 -05:00
f720fe0176
Merge pull request #114 from snshn/custom-network-timeout-option
...
Add option for custom network request timeout
2020-02-10 21:13:17 -05:00
727a5a410c
add option for custom network request timeout
2020-02-10 20:08:06 -05:00
5ef2b7c9dc
nullify JS within As' href attributes when needed
2020-02-03 01:47:35 -05:00
0e12cecd85
Revert "Merge pull request #106 from rhysd/proxy-support"
...
This reverts commit d8def879b2a9d114d04d
2020-01-21 13:01:22 +09:00
d47482fcd9
fix crash at setting empty values to HTTP proxies
...
with this patch `https_proxy=` and `http_proxy=` will work well.
2020-01-20 17:17:24 +09:00
b68624f2f3
support HTTP and HTTPS proxies ( fix #103 )
2020-01-20 17:02:43 +09:00
a9d114d04d
Merge pull request #105 from rhysd/refactor-main
...
Refactoring for main.rs to address several issues
2020-01-20 01:10:29 -05:00
4e4ebe9c98
refactor main to address several issues
...
Addressed issues:
- when specified URL is invalid, it exited successfully with doing
nothing. There was no way why it does not work for users
- it exited successfully even if invalid User-Agent value is specified
- it created file twice on `--output` option specified. It may cause an
issue when some file watcher (e.g. FsEvents on macOS) is watching
Improvements:
- handle errors with `Result::expect` consistently it correctly exits
with non-zero status on error
- define `Output` enum for handling both stdout and file outputs
2020-01-15 16:52:20 +09:00
1779f4a374
better comments for JS_DOM_EVENT_ATTRS constant
2020-01-15 14:33:27 +09:00
26e89ae6d3
use complete list of DOM event handlers
2020-01-15 13:58:09 +09:00
69d99b69e8
remove . in line comment
2020-01-13 23:47:07 +09:00
05985583f0
Switch timestamps from rfc822 local time to iso8601 UTC
2020-01-10 14:30:35 -05:00
651fa716b4
Clean user, pass, and fragment from URL before writing
2020-01-10 14:18:15 -05:00
67b79e92f9
simplify &x.into_iter() to x.iter()
2020-01-10 14:45:02 +09:00
b51f41fe34
trim attribute values
2020-01-10 14:41:05 +09:00
6f158dc6db
compare value of 'rel' properties in case-insensitive
2020-01-10 13:52:31 +09:00
8d7052b39c
ignore preload and prefetch sources
...
since all resources are embedded as data URL.
2020-01-09 18:18:21 +09:00
660511b8a0
define link type of <link> element as enum and prefer match statement
...
since match statement checks exhaustiveness
2020-01-09 16:55:42 +09:00
9be3982dc6
Added --no-context flag to disable adding context comment
2020-01-08 19:00:53 -05:00
27c9fb4cd3
Added comment indicating the context under which the page was downloaded
2020-01-08 18:51:18 -05:00
6e99ad13e7
upgrade reqwest to v0.10.0
...
This will improve build time and binary size as follows:
* Before
- **Compile targets**: 220
- **Build time**: `cargo build --release 1264.95s user 39.72s system 335% cpu 6:29.14 total`
- **Binary size**: 6578568 bytes
* After
- **Compile targets**: 170
- **Build time**: `cargo build --release 1130.64s user 32.15s system 359% cpu 5:23.69 total`
- **Binary size**: 6107088 bytes
* Differences
- **Compile targets**: 1.29x smaller
- **Build time**: 1.23x faster
- **Binary size**: 1.07x smaller
2020-01-07 14:22:32 +09:00
413dd66886
Merge pull request #96 from rhysd/refactorings
...
Refactorings
2020-01-05 18:46:31 -05:00
dc7ec6e7a8
remove more redundant type annotations
2020-01-04 16:33:11 +09:00
ed879231af
fix test code was broken by refactoring
2020-01-04 08:07:19 +09:00
ddf4b8ac13
prefer &str to String for reducing allocations
2020-01-04 08:05:02 +09:00
84c13f0605
prefer unwrap_or_default to unwrap_or
2020-01-04 07:58:29 +09:00
ce03e0e487
reduce allocation on checking DOM attributes and do not hard-code number of elements of array constant
...
`to_lower` allocates new string but the allocation is not necessary
here.
2020-01-04 07:52:47 +09:00
63e19998d0
reduce clones and fix some code styles and redundant code
2020-01-04 07:49:26 +09:00
75fb6961ed
migrate to Rust 2018
2020-01-03 00:33:49 +09:00
5ba8931502
Merge pull request #92 from snshn/output-file-option
...
Add option for saving output to file
2019-12-26 18:13:15 -05:00
88ffde0c3b
wipe integrity attributes
2019-12-26 09:44:01 -05:00
bfb97bd062
add option for saving output to file
2019-12-26 00:45:20 -05:00
295931041c
Merge pull request #80 from Alch-Emi/lazyload
...
Add support for lazy loaded images
2019-12-24 17:11:21 -05:00
dab4ae6965
Merged Y2Z/master with Alch-Emi/lazyload
2019-12-24 10:07:56 -05:00
c7fc121c7c
use clean URLs as hashmap keys
2019-12-18 11:49:38 -05:00
9ff9dd0928
Merge pull request #82 from snshn/str
...
implement str!() macro
2019-12-13 03:51:44 -05:00
3d4a932ac1
Merge Y2Z/master, fix conflicts between shared-client & resolve-css
2019-12-12 19:29:21 -05:00
9fe913d853
implement str!() macro
2019-12-11 01:36:14 -05:00
862489e41b
Get rid of brackets around URLs
2019-12-11 01:17:00 -05:00
322ab41b8c
Updated tests to reflect API changes
2019-12-10 00:00:15 -05:00
65d0eab793
Use a shared client initialized in main.rs
2019-12-09 22:17:54 -05:00
292221ea28
Lazyloaded images are now loaded at compilation, with placeholders omitted
2019-12-09 19:40:29 -05:00
614af44c92
Gramatical and stylistic fixes
2019-12-09 13:58:12 -05:00
feb37f5812
Added support for lazy loaded images
...
Note: The way this patch works is by resolving any data-src tags on images in
the same way as normal source tags are resolved. It is assumed that most
lazy-load libraries will use this tag, and that if this tag is set, then it is a
URL that is in use.
2019-12-06 19:27:41 -05:00
028beb821c
Rustfmt update for nightly formatter
2019-12-06 16:46:52 -05:00
76ccff80f9
Fixed failure of regex to match @imports
2019-12-06 16:15:34 -05:00
45335d7507
Support links in style= attributes
2019-12-06 15:28:08 -05:00
Sunshine
rhysd
Emi Simpson