ef3684025b
move to use http instead of https
2022-09-11 14:30:44 -04:00
db7ee697b3
rewrite small part of the input argument handling
...
the commit rewrite a small part of the input argument handling, trying
to follow besr rust practices.
We get rid of a variable and of a mutable reference while keeping the
code a bit more coincise.
2022-09-11 14:30:44 -04:00
900dd8d163
offload percent decoding to percent-encoding crate
2021-10-17 22:26:11 -10:00
2cc1870033
get rid of macros, move tests out of src
2021-10-16 21:16:37 -10:00
eeaea0df16
fix use of wrong charset
2021-07-02 21:35:06 -10:00
4bc8043f0f
account for charset when creating data URLs
2021-06-08 12:54:16 -10:00
125aeeec3b
improve validation of charset found in HTML, use genuinely infinite timeout
2021-06-08 11:50:46 -10:00
7686b2ea64
avoid excessive parsing of HTML into DOM
2021-06-08 03:57:28 -10:00
b29b9a6a7c
add option for saving document using custom encoding
2021-06-08 03:39:27 -10:00
b8aa545e8c
add support for wider range of charsets
2021-06-08 02:30:15 -10:00
a6ddf1c13a
simplify code responsible for processing CSS
2021-03-14 19:42:57 -10:00
2e86ee67a5
revamp codebase
2021-03-11 14:15:18 -10:00
4fa2eda983
make it possible to specify an empty user-agent string
2020-12-31 12:57:22 -10:00
870a4b150e
make possible to use stdin as input method
2020-12-25 21:23:29 -10:00
15d98a7269
don't modify base url by default, add option for setting it
2020-12-24 18:38:44 -10:00
38a6f963ad
account for network errors, add option to ignore them
2020-11-22 16:49:26 -10:00
1d6392cb28
implement support for BASE tag
2020-08-01 02:35:07 -04:00
73c35eaccb
treat minus for output target file path as stdout
2020-07-14 03:35:59 -04:00
09d41d2cf1
automatically obtain favicon.ico
2020-07-14 02:58:29 -04:00
bc98aca2a2
indent items in retrieval log to form a tree-like structure
2020-06-28 16:11:15 -04:00
ddb97009e9
pass options object instead of using separate parameters
2020-06-28 01:36:41 -04:00
2ac964fae5
include font-src into CSP
2020-06-26 18:14:46 -04:00
0347fd3985
move metadata tag code into a function
2020-06-25 18:23:56 -04:00
5c229c51da
move functions related to URL manipulation into url.rs
2020-06-24 03:16:40 -04:00
f9aac6f41b
update crates
2020-06-20 01:05:39 -04:00
4e31d0433e
only attempt to remove credentals from HTTP(S) URLs
2020-06-01 05:28:02 -04:00
a18df74946
refactor code and implement integrity validation
2020-05-12 02:51:37 -04:00
c4569343a4
improve metadata comments
2020-04-30 20:23:09 -04:00
c999359b9f
Merge branch 'context-comment' of github.com:Alch-Emi/monolith into context-comment
2020-04-30 19:54:13 -04:00
9b40dbbf27
add option to exclude web fonts
2020-04-22 09:11:20 -04:00
594ad55bd8
improve data URL media type detection
2020-04-10 05:50:33 -04:00
875481b9a2
update help dialog
2020-04-02 03:04:21 -04:00
4951fea730
implement full CSS parsing
2020-04-02 01:09:32 -04:00
59a8be493d
add support for working with local assets
2020-03-22 15:48:23 -04:00
0d1e21e9ad
add black box tests
2020-02-23 22:48:14 -05:00
3d2d40e7cd
add support for data URL targets
2020-02-23 22:25:37 -05:00
928664dc88
correct is_valid_url to is_http_url
2020-02-23 22:24:33 -05:00
727a5a410c
add option for custom network request timeout
2020-02-10 20:08:06 -05:00
0e12cecd85
Revert "Merge pull request #106 from rhysd/proxy-support"
...
This reverts commit d8def879b2a9d114d04d
2020-01-21 13:01:22 +09:00
d47482fcd9
fix crash at setting empty values to HTTP proxies
...
with this patch `https_proxy=` and `http_proxy=` will work well.
2020-01-20 17:17:24 +09:00
b68624f2f3
support HTTP and HTTPS proxies ( fix #103 )
2020-01-20 17:02:43 +09:00
4e4ebe9c98
refactor main to address several issues
...
Addressed issues:
- when specified URL is invalid, it exited successfully with doing
nothing. There was no way why it does not work for users
- it exited successfully even if invalid User-Agent value is specified
- it created file twice on `--output` option specified. It may cause an
issue when some file watcher (e.g. FsEvents on macOS) is watching
Improvements:
- handle errors with `Result::expect` consistently it correctly exits
with non-zero status on error
- define `Output` enum for handling both stdout and file outputs
2020-01-15 16:52:20 +09:00
05985583f0
Switch timestamps from rfc822 local time to iso8601 UTC
2020-01-10 14:30:35 -05:00
651fa716b4
Clean user, pass, and fragment from URL before writing
2020-01-10 14:18:15 -05:00
9be3982dc6
Added --no-context flag to disable adding context comment
2020-01-08 19:00:53 -05:00
27c9fb4cd3
Added comment indicating the context under which the page was downloaded
2020-01-08 18:51:18 -05:00
6e99ad13e7
upgrade reqwest to v0.10.0
...
This will improve build time and binary size as follows:
* Before
- **Compile targets**: 220
- **Build time**: `cargo build --release 1264.95s user 39.72s system 335% cpu 6:29.14 total`
- **Binary size**: 6578568 bytes
* After
- **Compile targets**: 170
- **Build time**: `cargo build --release 1130.64s user 32.15s system 359% cpu 5:23.69 total`
- **Binary size**: 6107088 bytes
* Differences
- **Compile targets**: 1.29x smaller
- **Build time**: 1.23x faster
- **Binary size**: 1.07x smaller
2020-01-07 14:22:32 +09:00
75fb6961ed
migrate to Rust 2018
2020-01-03 00:33:49 +09:00
bfb97bd062
add option for saving output to file
2019-12-26 00:45:20 -05:00
65d0eab793
Use a shared client initialized in main.rs
2019-12-09 22:17:54 -05:00
Simone Mosciatti
Sunshine
rhysd
Emi Simpson