Mirror/monolith
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
430 commits 1 branch 19 tags 4.2 MiB
Commit graph

58 commits

Author SHA1 Message Date
Sunshine
870a4b150e
make possible to use stdin as input method 2020-12-25 21:23:29 -10:00
Sunshine
15d98a7269
don't modify base url by default, add option for setting it 2020-12-24 18:38:44 -10:00
Sunshine
38a6f963ad
account for network errors, add option to ignore them 2020-11-22 16:49:26 -10:00
Sunshine
1d6392cb28
implement support for BASE tag 2020-08-01 02:35:07 -04:00
Sunshine
73c35eaccb
treat minus for output target file path as stdout 2020-07-14 03:35:59 -04:00
Sunshine
09d41d2cf1
automatically obtain favicon.ico 2020-07-14 02:58:29 -04:00
Sunshine
bc98aca2a2
indent items in retrieval log to form a tree-like structure 2020-06-28 16:11:15 -04:00
Sunshine
ddb97009e9
pass options object instead of using separate parameters 2020-06-28 01:36:41 -04:00
Sunshine
2ac964fae5
include font-src into CSP 2020-06-26 18:14:46 -04:00
Sunshine
0347fd3985
move metadata tag code into a function 2020-06-25 18:23:56 -04:00
Sunshine
5c229c51da
move functions related to URL manipulation into url.rs 2020-06-24 03:16:40 -04:00
Sunshine
f9aac6f41b
update crates 2020-06-20 01:05:39 -04:00
Sunshine
4e31d0433e
only attempt to remove credentals from HTTP(S) URLs 2020-06-01 05:28:02 -04:00
Sunshine
a18df74946
refactor code and implement integrity validation 2020-05-12 02:51:37 -04:00
Sunshine
c4569343a4
improve metadata comments 2020-04-30 20:23:09 -04:00
Sunshine
c999359b9f
Merge branch 'context-comment' of github.com:Alch-Emi/monolith into context-comment 2020-04-30 19:54:13 -04:00
Sunshine
9b40dbbf27
add option to exclude web fonts 2020-04-22 09:11:20 -04:00
Sunshine
594ad55bd8
improve data URL media type detection 2020-04-10 05:50:33 -04:00
Sunshine
875481b9a2
update help dialog 2020-04-02 03:04:21 -04:00
Sunshine
4951fea730
implement full CSS parsing 2020-04-02 01:09:32 -04:00
Sunshine
59a8be493d
add support for working with local assets 2020-03-22 15:48:23 -04:00
Sunshine
0d1e21e9ad
add black box tests 2020-02-23 22:48:14 -05:00
Sunshine
3d2d40e7cd
add support for data URL targets 2020-02-23 22:25:37 -05:00
Sunshine
928664dc88
correct is_valid_url to is_http_url 2020-02-23 22:24:33 -05:00
Sunshine
727a5a410c
add option for custom network request timeout 2020-02-10 20:08:06 -05:00
rhysd
0e12cecd85 Revert "Merge pull request #106 from rhysd/proxy-support" 
This reverts commit d8def879b2, reversing
changes made to a9d114d04d.
 2020-01-21 13:01:22 +09:00
rhysd
d47482fcd9 fix crash at setting empty values to HTTP proxies 
with this patch `https_proxy=` and `http_proxy=` will work well.
 2020-01-20 17:17:24 +09:00
rhysd
b68624f2f3 support HTTP and HTTPS proxies (fix #103) 2020-01-20 17:02:43 +09:00
rhysd
4e4ebe9c98 refactor main to address several issues 
Addressed issues:

- when specified URL is invalid, it exited successfully with doing
  nothing. There was no way why it does not work for users
- it exited successfully even if invalid User-Agent value is specified
- it created file twice on `--output` option specified. It may cause an
  issue when some file watcher (e.g. FsEvents on macOS) is watching

Improvements:
- handle errors with `Result::expect` consistently it correctly exits
  with non-zero status on error
- define `Output` enum for handling both stdout and file outputs
 2020-01-15 16:52:20 +09:00
Emi Simpson
05985583f0
Switch timestamps from rfc822 local time to iso8601 UTC 2020-01-10 14:30:35 -05:00
Emi Simpson
651fa716b4
Clean user, pass, and fragment from URL before writing 2020-01-10 14:18:15 -05:00
Emi Simpson
9be3982dc6
Added --no-context flag to disable adding context comment 2020-01-08 19:00:53 -05:00
Emi Simpson
27c9fb4cd3
Added comment indicating the context under which the page was downloaded 2020-01-08 18:51:18 -05:00
rhysd
6e99ad13e7 upgrade reqwest to v0.10.0 
This will improve build time and binary size as follows:

* Before

- **Compile targets**: 220
- **Build time**: `cargo build --release  1264.95s user 39.72s system 335% cpu 6:29.14 total`
- **Binary size**: 6578568 bytes

* After

- **Compile targets**: 170
- **Build time**: `cargo build --release  1130.64s user 32.15s system 359% cpu 5:23.69 total`
- **Binary size**: 6107088 bytes

* Differences

- **Compile targets**: 1.29x smaller
- **Build time**: 1.23x faster
- **Binary size**: 1.07x smaller
 2020-01-07 14:22:32 +09:00
rhysd
75fb6961ed migrate to Rust 2018 2020-01-03 00:33:49 +09:00
Sunshine
bfb97bd062 add option for saving output to file 2019-12-26 00:45:20 -05:00
Emi Simpson
65d0eab793
Use a shared client initialized in main.rs 2019-12-09 22:17:54 -05:00
voila
1ff5e91087 Use HashMap as cache to minimize the number of HTTP requests (#75) 
Use HashMap as cache to minimize the number of HTTP requests
 2019-10-22 18:33:22 -04:00
robatipoor
2e48ea90e1 move argument parser section to args mod 2019-10-10 08:58:12 +03:30
Sunshine
0896f2e214 Properly handle 30x redirects 2019-09-30 23:58:09 -04:00
Vincent Flyson
3948ea3aa0 Improve code structure 2019-09-29 17:15:49 -04:00
Vincent Flyson
eec05767cf Add support for poster attribute 2019-09-22 12:57:50 -04:00
Vincent Flyson
88a230872c Add CSP isolation, no CSS, and no iframe options 2019-09-21 22:59:03 -04:00
Vincent Flyson
02b717ae54 Add flag to ignore errors related to TLS certificates 2019-08-26 23:17:36 -04:00
Vincent Flyson
2b96f9a32a Add flag for silent mode 2019-08-25 11:41:30 -04:00
Vincent Flyson
d8d6437a15
Merge branch 'master' into author-robatipoor 2019-08-23 23:24:07 -04:00
Vincent Flyson
c34d77d5d8 Revamp resolve_url() and improve code format 2019-08-23 23:06:14 -04:00
Vincent Flyson
2be5b1c235 Add support for iframes 2019-08-23 20:16:16 -04:00
Vincent Flyson
13429e32d3 Allow HTTP redirects and preserve email links 2019-08-23 16:00:05 -04:00
Vincent Flyson
d3497b5db1
Merge branch 'master' into author-robatipoor 2019-08-23 15:09:46 -04:00