webdump.1 - webdump - HTML to plain-text converter for webpages
HTML git clone git://git.codemadness.org/webdump
DIR Log
DIR Files
DIR Refs
DIR README
DIR LICENSE
---
webdump.1 (4489B)
---
1 .Dd July 15, 2026
2 .Dt WEBDUMP 1
3 .Os
4 .Sh NAME
5 .Nm webdump
6 .Nd convert HTML to plain-text
7 .Sh SYNOPSIS
8 .Nm
9 .Op Fl 8adiIlorx
10 .Op Fl b Ar baseurl
11 .Op Fl s Ar selector
12 .Op Fl u Ar selector
13 .Op Fl w Ar termwidth
14 .Sh DESCRIPTION
15 .Nm
16 reads UTF-8 HTML data from stdin.
17 It converts and writes the output as plain-text to stdout.
18 A
19 .Ar baseurl
20 can be specified if the links in the feed are relative URLs.
21 This must be an absolute URI.
22 .Pp
23 The options are as follows:
24 .Bl -tag -width Ds
25 .It Fl 8
26 Use UTF-8 symbols for certain items like bullet items and rulers to make the
27 output fancier.
28 .It Fl a
29 Toggle ANSI escape codes.
30 This is used to markup text with bold, italic, underline, etc, by default it is
31 not enabled.
32 .It Fl b Ar baseurl
33 Base URL of links.
34 This is used to make links absolute.
35 The specified URL is always preferred over the value in a <base/> tag.
36 .It Fl d
37 Deduplicate link references.
38 When a duplicate link reference is found reuse the same link reference number.
39 .It Fl i
40 Toggle if link reference numbers are displayed inline or not, by default it is
41 not enabled.
42 .It Fl I
43 Toggle if URLs for link reference are displayed inline or not, by default it is
44 not enabled.
45 .It Fl l
46 Toggle if link references are displayed at the bottom or not, by default it is
47 not enabled.
48 .It Fl o
49 Toggle OSC 8 escape sequence to mark inline hyperlinks, by default it is not
50 enabled.
51 This feature can be insecure in some terminals.
52 .It Fl r
53 Toggle if line-wrapping mode is enabled, by default it is not enabled.
54 This uses the terminal width which can be specified with the
55 .Fl w
56 option.
57 .It Fl s
58 CSS-like selectors, this sets a reader mode to show only content matching the
59 selector, see the section
60 .Sx SELECTOR SYNTAX
61 for the syntax.
62 Multiple selectors can be specified by separating them with a comma.
63 .It Fl u
64 CSS-like selectors, this sets a reader mode to hide content matching the
65 selector, see the section
66 .Sx SELECTOR SYNTAX
67 for the syntax.
68 Multiple selectors can be specified by separating them with a comma.
69 .It Fl w Ar termwidth
70 The terminal width.
71 The default is 77 characters.
72 .It Fl x
73 Extract and write each resource (such as links and images) to file
74 descriptor 3.
75 The items are output per line in a TAB-separated format.
76 The fields are: type TAB v|h TAB url NEWLINE.
77 v or h indicates a visible or hidden element.
78 .El
79 .Sh SELECTOR SYNTAX
80 Some inspiration was taken from the CSS syntax, but it is more limited.
81 Tag names are matched case-insensitive.
82 Classname and ID values are matched case-sensitive.
83 .Pp
84 Some examples:
85 .Bl -item
86 .It
87 "main" would match on the "main" tags.
88 .It
89 "#someid" would match on any tag which has the id attribute set to "someid".
90 .It
91 ".someclass" would match on any tag which has the class attribute set to
92 "someclass".
93 .It
94 "main#someid" would match on the "main" tag which has the id attribute set to
95 "someid".
96 .It
97 "main.someclass" would match on the "main" tags which has the class
98 attribute set to "someclass".
99 .It
100 "ul li" would match on any "li" tag which also has a parent "ul" tag.
101 .It
102 "li@0" would match on any "li" tag which is also the first child element of its
103 parent container.
104 Note that this differs from filtering on a collection of "li" elements.
105 .El
106 .Sh EXIT STATUS
107 .Ex -std
108 .Sh EXAMPLES
109 Fetch a webpage using
110 .Xr curl 1
111 and pipe the output to
112 .Nm
113 enabling line-wrapping and writing output to the pager
114 .Xr less 1 :
115 .Bd -literal
116 url='https://codemadness.org/sfeed.html'
117 curl -s "$url" | webdump -r -b "$url" | less
118 .Ed
119 .Pp
120 Pipe the HTML output to
121 .Nm
122 enabling many options (unicode and markup, line-wrapping, link references).
123 Transform relative URLs to absolute URLs using the base URL.
124 Then pipe it to
125 .Xr less 1
126 with the
127 .Fl R
128 option to enable ANSI escape codes in this pager:
129 .Bd -literal
130 curl -s "$url" | webdump -8 -a -i -l -r -b "$url" | less -R
131 .Ed
132 .Pp
133 Use a simple selector using the
134 .Fl s
135 option to only show content in the "main" tag:
136 .Bd -literal
137 curl -s "$url" | webdump -s 'main' -8 -a -i -l -r -b "$url" | less -R
138 .Ed
139 .Pp
140 To write resources as TAB-separated lines from file descriptor 3 to a separate
141 file while ignoring the regular standard output:
142 .Bd -literal
143 curl -s "$url" | webdump -x -b "$url" 3>urls >/dev/null
144 .Ed
145 .Pp
146 To use
147 .Nm
148 as a HTML to text filter for example in the mutt mail client, change in
149 ~/.mailcap:
150 .Bd -literal
151 text/html; webdump -i -l -r < %s; needsterminal; copiousoutput
152 .Ed
153 .Sh SEE ALSO
154 .Xr curl 1 ,
155 .Xr xmllint 1 ,
156 .Xr xmlstarlet 1 ,
157 .Xr ftp 1
158 .Sh AUTHORS
159 .An Hiltjo Posthuma Aq Mt hiltjo@codemadness.org