quark, branch HEAD

Fix buffer over-read in decode()

2022-08-16T20:37:50Z

commit 5ad0df91757fbc577ffceeca633725e962da345d parent a982fa636704a436c3d1016b1f82806f607b7556 Author: HushBugger Date: Tue, 16 Aug 2022 22:37:50 +0200 Fix buffer over-read in decode() The format specifier for parsing percent-formatted characters uses a maximum number of digits, not an exact number of digits. If the hex number has only one digit this will skip a character, potentially pointing past the terminating null byte.

Fix strftime error handling

2022-07-08T18:12:17Z

commit a982fa636704a436c3d1016b1f82806f607b7556 parent bc9ba3e52e6ab56bb0761f95750c2c3be3012b52 Author: robert Date: Fri, 8 Jul 2022 11:12:17 -0700 Fix strftime error handling Unlike snprintf, strftime buffer contents are undefined when it fails, so make sure the buffer is null-terminated. To prevent garbage from being printed out, we simply set the timestamp to the empty string, but maybe setting it to "unknown time" or something similar would be better. Either way, I don't think this can fail until year 10000, so it's not a big deal.

Remove superfluous byteorder conversion

2022-04-19T10:20:40Z

commit bc9ba3e52e6ab56bb0761f95750c2c3be3012b52 parent eead7a5fd2cbe5aa54d278c8171bd9476f91b2e4 Author: Thomas Oltmann Date: Tue, 19 Apr 2022 12:20:40 +0200 Remove superfluous byteorder conversion When comparing IPv4 addresses in sock_same_addr() we don't need to correct their byteorder just to see if they are equal or not. Byte swapping would only be needed if we needed to know which address had the greater value.

Fix inverted conditional in sock_same_addr()

2022-04-19T10:04:57Z

commit eead7a5fd2cbe5aa54d278c8171bd9476f91b2e4 parent 68b4f733b2755762e43df90f73db5a6ec8d14104 Author: Thomas Oltmann Date: Tue, 19 Apr 2022 12:04:57 +0200 Fix inverted conditional in sock_same_addr() sock_same_addr() is supposed to return 0 if sa1 and sa2 are different addresses. Since memcmp() returns 0 if its arguments are equal, we need to flip the return value by comparing it to 0.

Improve connection logging

2021-02-22T18:06:36Z

commit 68b4f733b2755762e43df90f73db5a6ec8d14104 parent 7814309e9a2c386f646892403e53f2a87b929b0c Author: Laslo Hunhold Date: Mon, 22 Feb 2021 19:06:36 +0100 Improve connection logging Given connection_log() can be called on "incomplete" connections since the last changes regarding properly logging dropped connections, the function sock_get_inaddr_str() had the broken assumption that when ss_family was neither AF_INET nor AF_INET6 it would be AF_UNIX, leading to the wrong logging of just-accepted-and-dropped connections as "uds", even when listening on an IP socket. As a solution, the switch is amended and yields the expected in-address-string "-" when the given sockaddr-struct is not filled yet. Signed-off-by: Laslo Hunhold

Use one listening socket for all threads

2021-02-22T17:39:49Z

commit 7814309e9a2c386f646892403e53f2a87b929b0c parent e2463e733e4880c1d8034e3b90072825509ceb69 Author: Laslo Hunhold Date: Mon, 22 Feb 2021 18:39:49 +0100 Use one listening socket for all threads Previously, we employed SO_REUSEPORT and bound as many listening sockets as we had threads. Inside the kernel, this creates separate queues for each socket and a hash-based-round-robin distributes incoming requests evenly among the listening sockets. As a result, the load was well-balanced among threads, in contrast to sharing one listening socket (and one shared queue), because when epoll() registers new incoming connections, it takes the last-activated thread (FIFO). As a result, only one thread usually gets most of the load, unless the server is really stressed. So why change it back? The reason is latency. It's not a coincidence that when you're at the supermarket you very often see another queue pass along much faster than you. Even though a congestion (i.e. slow cashier or customer) is completely random and no cashier-post is favoured, all people in the congested queue are affected (the chance is 2/3 to be in a slower queue when there are 3 queues). A much more efficient approach is to have one shared queue and 3 cashiers. Even when there's a congestion at one cashier-post, the others will continue processing customers and the overall latency is much more consistent. The same applies to the connection-queue: If a thread is really busy with a request, the waiting connections will be affected without the possibility of being processed by another idle thread. Another reason is efficiency: The hash-based-round-robin in the kernel adds a small overhead that can be avoided this way, it removes complexity in the code and undoes the inconsistent "hack" where we just gave each thread the same listening socket when we were working with a UNIX-domain socket. This change adds a small "regression": When quark is hammered with a lot of connections (>15k/s), it is more motivated to drop connections because epoll has the aforementioned FIFO-behaviour (i.e. it tends to pass the connection-events to the same thread), which usually accepts before processing read/write-related events. This in turn exhausts the thread's connection-pool at some point, but because it handles that well, we never enter a DoS-state. This commit is a preparation for upcoming changes. Signed-off-by: Laslo Hunhold

Refactor server- and connection-logic into separate components

2021-02-15T17:25:33Z

commit e2463e733e4880c1d8034e3b90072825509ceb69 parent 2920ba56d92b461c39f3c7c9c5a264f38b899fd7 Author: Laslo Hunhold Date: Mon, 15 Feb 2021 18:25:33 +0100 Refactor server- and connection-logic into separate components The server-part (creating a pool, launching workers, etc.) and methods concerning the handling of connections (accepting, serving, logging, etc.) were all in main.c, making it unnecessarily complicated on top of the fact that there is already a lot to do there. Now, these parts are moved into server.h/server.c and connection.h/ connection.c respectively, while main() now just only has to call into server_init_thread_pool(). In other words, main() now only has to deal with the argument parsing and preparations, and the rest is done by other program parts. The methods are prefixed with server_* and connection_* respectively, making it much easier to see where each method is defined in the program. This refactoring also separates concerns well. The server-struct is now in server.h (and not in util.h, where it was always out of place) and spacetok() is moved into util.c. Signed-off-by: Laslo Hunhold

Further refine dropout-candidates over response-type and progress

2021-02-13T10:46:59Z

commit 2920ba56d92b461c39f3c7c9c5a264f38b899fd7 parent 6d7c7b6ff701fafc2a649b21a66a92a9ab626221 Author: Laslo Hunhold Date: Sat, 13 Feb 2021 11:46:59 +0100 Further refine dropout-candidates over response-type and progress There might not only be scenarios where the server is under attack by a single entity, but possibly a botnet of hundreds of computers with a few connections each. An honest client connecting to the server with a few connections that might include a long-running download might end up as the client with the most simultaneous connections (e.g. 6 over the respective 5 of each botnet-member). If we end up dropping one of the honest client's connections, we don't want it to be the long-running-download, but lower-priority things (directory listings, error responses, not-far-advanced downloads; in increasing order of importance). Among similar types, we drop those that are not as far advanced in absolute terms. The reasoning behind this (and against relative progress) is that it's worse to drop a download at 50% when the requested file is 300MB compared to dropping another connection at 75% when the requested file is merely a few kB large. Signed-off-by: Laslo Hunhold

Apply (D)DoS-Hardening

2021-02-06T00:15:53Z

commit 6d7c7b6ff701fafc2a649b21a66a92a9ab626221 parent 8aa213c123cbcfc2e50a77142c2206c202705e90 Author: Laslo Hunhold Date: Sat, 6 Feb 2021 01:15:53 +0100 Apply (D)DoS-Hardening Until now, if quark found that in case of an incoming connection it didn't have any vacant connection slots left, it would just not accept() and thus block any further new connections until a slot was free. This may sound reasonable at first, but there are cases where the connected clients are not benevolent and might firmly occupy all slots with simple request flooding or more advanced attacks like slowloris, R-U-Dead-Yet or Slow Read, which all boil down to sending or receiving data really slowly. The latter attacks are very effective and require very little resources on the attacker's side. Thus, the only way to keep things going is to bite the bullet and always accept a connection, even if it means dropping another connection for it. In this case, an algorithm determines which client has the most connections and drops the least-advanced connection of it (i.e. a connection in the earliest possible stage). Stress-tests with slowhttptest[0] and siege yielded excellent results, where quark always remained responsive for the normal visitor despite an active massive DoS-attack using the aforementioned methods. Side-note: In the spirit of not having any memory allocations in quark during "runtime", the constant-space algorithm used to determine the client with the most connections is quadratic in time over the number of slots. This, however, is not a big deal, given the number of slots is always relatively small and such an out-of-connection-scenario is an edge-case. [0]:https://github.com/shekyan/slowhttptest [1]:https://github.com/JoeDog/siege Signed-off-by: Laslo Hunhold

Refactor resource-parsing and -handling heavily

2021-02-03T17:33:51Z

commit 8aa213c123cbcfc2e50a77142c2206c202705e90 parent f45ca668af3d615e9215db49f190ea2833cecc18 Author: Laslo Hunhold Date: Wed, 3 Feb 2021 18:33:51 +0100 Refactor resource-parsing and -handling heavily The previous approach of stripping query and fragment from the resource was flawed and fixing this issue motivated a much deeper refactor that was on my todo-list for a while. First off, the resource-request (e.g. "/projects/index.html?123#eme") is made up of the path ("/projects/index.html") and optionally the query ("123") and fragment ("eme"). Instead of trying to break it up or stripping it in the response-generation, we now store these three things separately in the request-struct. Calling the resource-request an "URI" was wrong, as the URI also includes the authority (i.e. the host in this case) and the protocol-prefix (i.e. "http://"). To fix this, the respective fields in the request- and response-structs had to be renamed, as follows: This commit adds a differentiation between a "path" (what is publicly requested) and an "internal_path" (what we actually serve in the file system). These two things can differ, e.g. with virtual hosts. The cleanup and generation of these paths for the response-struct is heavily refactored in http_prepare_response(), eliminating some deep bugs that were due to the previously complicated approach. Instead of doing everything by hand and having a very complicated logic on when, after the path-cleanup, it was necessary to do a redirect, the cleanup sections are exported to separate functions which indicate when a redirect is necessary. This also makes more complex path-processing possible, if desired, and definitely increases the readability. In total, this makes the path-processing much more straightforward, and fixes the problem where a redirect would strip queries and fragments. Possible file-access problems for virtual hosts were also fixed while also eliminating the pretty hacky RELPATH-macro, among other things. Signed-off-by: Laslo Hunhold

Make queue-event-error-detection stricter

2021-02-02T21:20:03Z

commit f45ca668af3d615e9215db49f190ea2833cecc18 parent 35e3b69d60f337724163e9543b5728b907ce34dd Author: Laslo Hunhold Date: Tue, 2 Feb 2021 22:20:03 +0100 Make queue-event-error-detection stricter Everything which is not a pollin or pollout is now considered an error. This is due to the fact how variable epoll(7) is in regard to possible events, often depending on your kernel configuration (see for instance EPOLLPRI and possible future additions). In this context, we also rename the function to better reflect its purpose. Signed-off-by: Laslo Hunhold

Refactor queue-event-handling to be pointer-centric

2021-01-31T10:33:46Z

commit 35e3b69d60f337724163e9543b5728b907ce34dd parent f1892c4dffa7df6995cd8c9ebd47e0ce7f0e457e Author: Laslo Hunhold Date: Sun, 31 Jan 2021 11:33:46 +0100 Refactor queue-event-handling to be pointer-centric This is one aspect where we can see that kqueue(2) is superior to epoll(7). While kqueue(2) allows you to both store a pointer and read out the fd of an event in the ready-list, epoll(7) only provides you with one of the two (by only offering a union-type-field where you either store an fd or a pointer). Previously, wrapper functions would allow to extract either from any event, which was only non-hacky in the OpenBSD-case, because nothing stopped you from reading the "fd" from the union that actually was assigned as a pointer. The distinction was there in the first place because we are listening on the main incoming socket and other "client"- sockets at the same time. The main-socket-events contained the socket-fd and the client-sockets contained the connection-struct-pointer. Now, we just set the data-pointer of the main socket to NULL to indicate it, yielding a much more refined usage with one fewer way to screw things up. Signed-off-by: Laslo Hunhold

Fix handling of unexpected hangups

2021-01-30T23:39:11Z

commit f1892c4dffa7df6995cd8c9ebd47e0ce7f0e457e parent 3729e7222aafc4c4ca30351748a89e05f78e2230 Author: Laslo Hunhold Date: Sun, 31 Jan 2021 00:39:11 +0100 Fix handling of unexpected hangups During slowloris-stress-testing I noticed that closing the tool would pretty reliably trigger a case where quark was continuously getting edge-triggered on EPOLLIN and notified that there was something to read on the now definitely closed client-sockets. This was wrong and due to the fact that I mishandled the case when read() returns 0 in http_recv_header(). The condition read()==0 and EPOLL or EWOULDBLOCK does not indicate that you should try again. It's an EOF-condition and should be handled as such. Given the request is incomplete at this point, this is handled as a bad request (HTTP status 400) mostly for the logs at this point, as being able to transmit the error page itself is highly unlikely. Signed-off-by: Laslo Hunhold

Return -1 in case of errors in queue event wrapper functions.

2021-01-30T12:54:58Z

commit 3729e7222aafc4c4ca30351748a89e05f78e2230 parent 319ba7083fdde836d6614c6b8b228bf3a9849e95 Author: Rainer Holzner Date: Sat, 30 Jan 2021 13:54:58 +0100 Return -1 in case of errors in queue event wrapper functions. Use same data type for nready (number of events) as returned by queue_wait().

Ignore queries and fragments in URIs

2021-01-30T11:53:00Z

commit 319ba7083fdde836d6614c6b8b228bf3a9849e95 parent c6a9055e5a30be570e30da8d216c39662c3a3f99 Author: Laslo Hunhold Date: Sat, 30 Jan 2021 12:53:00 +0100 Ignore queries and fragments in URIs Previously, a request for "/index.html" would yield a 200, while a request for "/index.html?foo=bar" would yield a 404, as quark would look for the file "index.html?foo=bar" in the serve directory. To accomodate this behaviour, it's no longer sufficient to just compare realuri and req->uri. Instead, we set a "dirty" flag every time we change the URI in such a way that it requires a redirect. According to RFC 3986 section 3, queries and fragments are there to (further) "identify a resource within the scope of the URI's scheme and naming authority (if any)". However, it's perfectly legitimate to just ignore this further specification when the URI itself is already pointing at a unique resource (i.e. "/index.html"). This behaviour is consistent with dynamic web applications which usually ignore parameters they don't care about. Quark is too much Zen to care about any parameters. This has the added bonus that you can now clone repositories (read-only) via the "dumb" HTTP git-protocol, so git clone https://example.org/git/project.git is now possible (provided you run update-server-info during the post-update-hook). This wouldn't work previously because git, when asked to clone via HTTP, would first probe the server with a request for project.git/info/refs?service=git-upload-pack (i.e. asking for the "smart" HTTP git-protocol to confirm). Quark would return a 404, though, while git only gracefully "downgrades" to the "dumb" HTTP git-protocol if the request succeeds but only yields a basic 200 response without special git-headers. This way, it is now trivial to also share git-repositories (and other gracefully-downgrading protocols). While the "dumb" HTTP git-protocol only supports read-only-access, I don't think that's much of an overall loss (to the contrary!). HTTP authentication is broken and it makes much more sense to enable ssh-access to contributors and make them push changes via ssh. The key advantage of HTTP-cloning over git://-cloning is the fact that the git protocol can be tampered with, while the HTTP-protocol can be encapsulated into a secure TLS connection. Signed-off-by: Laslo Hunhold

Update LICENSE

2021-01-24T20:44:23Z

commit c6a9055e5a30be570e30da8d216c39662c3a3f99 parent 67c29aaba8a8194685677586338688e82c619e93 Author: Laslo Hunhold Date: Sun, 24 Jan 2021 21:44:23 +0100 Update LICENSE Signed-off-by: Laslo Hunhold

Make sure the docindex' mime-type is determined correctly

2021-01-24T20:38:38Z

commit 67c29aaba8a8194685677586338688e82c619e93 parent 8afc6416647585ec2695d57eee7c226216e4111c Author: Laslo Hunhold Date: Sun, 24 Jan 2021 21:38:38 +0100 Make sure the docindex' mime-type is determined correctly Previously, the docindex would always be served with the default mime-type, given the directory-URI does not have a file extension. Signed-off-by: Laslo Hunhold

Put docindex-appending into an else-clause

2021-01-24T20:35:11Z

commit 8afc6416647585ec2695d57eee7c226216e4111c parent deeec27c56d8f5049abac0dad3782f5daf95a1a3 Author: Laslo Hunhold Date: Sun, 24 Jan 2021 21:35:11 +0100 Put docindex-appending into an else-clause The previous if-branch leaves (goto, return) in all cases, making no functional difference, but this improves the explicitness. Signed-off-by: Laslo Hunhold

http: fix default index serving

2020-01-17T13:41:55Z

commit deeec27c56d8f5049abac0dad3782f5daf95a1a3 parent a4ea7cbe676adffd1dbd98b2bb7f68591b24d46c Author: Quentin Rameau Date: Fri, 17 Jan 2020 14:41:55 +0100 http: fix default index serving The previous code would find and stat the default index file, but would not append it to the file served, resulting in giving back a 0-length content but with a Content-Length header of the size of the index.

Revert "Fix directory-index-handling"

2021-01-24T20:14:13Z

commit a4ea7cbe676adffd1dbd98b2bb7f68591b24d46c parent 87ae2e9212c5cc7309eefa2a3f49a758862db6c7 Author: Laslo Hunhold Date: Sun, 24 Jan 2021 21:14:13 +0100 Revert "Fix directory-index-handling" This reverts commit 87ae2e9212c5cc7309eefa2a3f49a758862db6c7, which did three things at once and lacked proper git-author-attribution for Quentin for the underlying change. I have read too many OpenBSD-CVS-logs where this is common, but only due to the fact that CSV doesn't distinguish between author and committer, which leads to an antipattern in git.

Fix directory-index-handling

2021-01-24T17:37:03Z

commit 87ae2e9212c5cc7309eefa2a3f49a758862db6c7 parent f3b6d5efc375bedd287897dcaabffec7f9222ea6 Author: Laslo Hunhold Date: Sun, 24 Jan 2021 18:37:03 +0100 Fix directory-index-handling Previously, quark would properly check for the docindex-path, but not actually change the response-struct accordingly. I missed this during the restructurization of the code. Thanks to Quentin Rameau for spotting this issue! This commit is based on his patch. Signed-off-by: Laslo Hunhold

Properly set resource limits

2021-01-20T23:58:01Z

commit f3b6d5efc375bedd287897dcaabffec7f9222ea6 parent 959c855734e3af12f35532d76deb1ab85474f8f4 Author: Laslo Hunhold Date: Thu, 21 Jan 2021 00:58:01 +0100 Properly set resource limits Quark sets two rlimits, the maximum number of open file descriptors and the maximum number of threads. I made a mistake in the calculation of the former (forgetting about slots) and gave it a bit more headroom so we don't run into problems. Given the number open file descriptors is per-process, this can be considered done. The thread-limit is a different matter, because it's per user. To work around this, the program tries increasing the per-user-limit by the number of threads needed. If this hits the upper bound imposed by the system, we ignore this though, and just carry on. Sadly the getrlimit() errnos are not as insightful as I'd wish, because it uses EPERM for a lot of things other than excessive limits, but this is a good compromise. If we fail because of CAP_SYS_RESOURCE, which might remain uncaught previously in case setrlimit on RLIMIT_NOFILE lowers both limits, it will remain unreported here. However, I wouldn't want to spam the command line with such an error message when it's only a very high preset limit by a user triggering a kernel-limit-overflow. In case it is a lack of CAP_SYS_RESOURCE, this is later reported when pthread_create() fails, so we cover this case as well and thus give the user proper feedback on what he can do. Signed-off-by: Laslo Hunhold

Fix compilation on OpenBSD

2021-01-17T12:22:53Z

commit 959c855734e3af12f35532d76deb1ab85474f8f4 parent a9164839e6f091ff66b6684c65d55ed7f5a09ebb Author: Laslo Hunhold Date: Sun, 17 Jan 2021 13:22:53 +0100 Fix compilation on OpenBSD The OpenBSD-code was written "blindly" only with the manuals. The errors that occured are now fixed. It shows how well-written the OpenBSD manuals are that you can write such a nontrivial state-machine and it just works (once the syntax-errors are fixed). Signed-off-by: Laslo Hunhold

Also add a proper warning with a hint when bind() fails

2021-01-17T12:19:45Z

commit a9164839e6f091ff66b6684c65d55ed7f5a09ebb parent ed712285707ecba554d718ef28ec2fd83fb846be Author: Laslo Hunhold Date: Sun, 17 Jan 2021 13:19:45 +0100 Also add a proper warning with a hint when bind() fails Signed-off-by: Laslo Hunhold

Update license

2021-01-17T11:34:23Z

commit ed712285707ecba554d718ef28ec2fd83fb846be parent e5db41118f5c9bfc27338a803d6d4eebec05cc1b Author: Laslo Hunhold Date: Sun, 17 Jan 2021 12:34:23 +0100 Update license Signed-off-by: Laslo Hunhold

Improve permission-error-reporting and raise open-file-limit

2021-01-16T16:58:16Z

commit e5db41118f5c9bfc27338a803d6d4eebec05cc1b parent 7d26fc695d548b5a73305a97dce274a313e0f602 Author: Laslo Hunhold Date: Sat, 16 Jan 2021 17:58:16 +0100 Improve permission-error-reporting and raise open-file-limit There was a small bug, namely that when quark was executed as a normal user, the fork would fail because setrlimit() had made it impossible beforehand (given it only controls thread-counts). Now the setrlimit() for thread-count is done correctly after the fork. Additionally, given we also open at least as many files as we have threads (and each thread needs to keep multiple fd's open at the same time), we also now set the open-file-limit properly. Previously, when someone tried to execute quark as non-root, they would get the error message $ quark -p 5000 quark: fork: Resource temporarily unavailable $ This was due to the aforementioned "bug", but even still, they would've gotten an error message relating to a failed chroot. In either case, it might've been a bit confusing, which is why it now shows a clear error message on what's wrong and a possible "mitigation" (using capabilities(7) as an alternative to setuid or root): $ quark -p 5000 quark: You need to run as root or have CAP_SYS_CHROOT set $ CAP_SYS_CHROOT alone is not sufficient to run quark, and it will print further errors until all permissions are met, but I won't add a separate error handling and logic just to appease with a cumulative error-message. When trying to bind to a privileged port, you get $ quark -p 1000 quark: You need to run as root or have CAP_NET_BIND_SERVICE set to bind to privileged ports $ instead of $ quark -p 1000 quark: bind: Permission denied $ which is also a net-benefit. In this context, this commit also improves the error-reporting when someone tries 'dropping' to the root user or group by checking this beforehand and not with a getuid() and getgid() later on. Signed-off-by: Laslo Hunhold

Prevent overflow in strtonum()-parameters

2020-11-01T00:47:11Z

commit 7d26fc695d548b5a73305a97dce274a313e0f602 parent dff98c0bcaef7be220c563ebaebd66f8c6704197 Author: Laslo Hunhold Date: Sun, 1 Nov 2020 01:47:11 +0100 Prevent overflow in strtonum()-parameters Make sure not to overflow the long long value. Given the standard doesn't bring any tangible guarantees for the upper limits of size_t, we just determine which (long long or size_t) is larger at compile time. Thanks José Miguel Sánchez García for reporting this! Signed-off-by: Laslo Hunhold

Use epoll/kqueue and worker threads to handle connections

2020-10-31T23:27:46Z

commit dff98c0bcaef7be220c563ebaebd66f8c6704197 parent 4d3a6c5297015285f88a56cc47f6a53c372b1506 Author: Laslo Hunhold Date: Sun, 1 Nov 2020 00:27:46 +0100 Use epoll/kqueue and worker threads to handle connections This adds quite a bit of code, but is the culmination of the previous restructurizations. Each worker thread has a connection pool and the interesting part is that it's 100% nonblocking. If reading or writing blocks at any point, the worker thread can just drop it and continue with something else. This is especially powerful against attacks like slow loris, which cannot be caught with a forking-model and could easily be used in a DoS against a quark instance. There are no memory allocations at runtime, unless you use dirlistings, whose libc-allocations you can't work around. In case the connection pool is exhausted due to a lot of slow lorises, we still hit a DoS, but at least it can now be possible to assess the connection pool and just drop another connection that can be heuristically assessed as a "malicious" one (e.g. many connections from one client, long time in one state or something using a monotonic clock). Given we still sadly don't have kqueue in linux, which is 1000x times better than epoll, which is deeply flawed, I wrote a very thin wrapper in queue.{c,h} which exposes the necessary functions in a common interface. Signed-off-by: Laslo Hunhold

Prepare http_send_buf() http_recv_header() for blocking I/O

2020-10-31T23:10:54Z

commit 4d3a6c5297015285f88a56cc47f6a53c372b1506 parent 5d0221dd68c0d2b8796479d06b602be666d0f4c6 Author: Laslo Hunhold Date: Sun, 1 Nov 2020 00:10:54 +0100 Prepare http_send_buf() http_recv_header() for blocking I/O Signed-off-by: Laslo Hunhold

Introduce state-handling and interruptiblity in serve()

2020-09-16T16:18:40Z

commit 5d0221dd68c0d2b8796479d06b602be666d0f4c6 parent e67eddd517d3f20a24ee14e1eab92806ce5bb20c Author: Laslo Hunhold Date: Wed, 16 Sep 2020 18:18:40 +0200 Introduce state-handling and interruptiblity in serve() Granted, the changes won't make sense right now, but this allows the functions http_recv_header() and http_send_buf() to be interrupted (e.g. if a read or write blocks). In the main loop, we currently just have a do-while-loop, which is not even necessary because the sockets are blocking and the aforementioned two functions will always complete their task, unless there's an error. This change also adapts the code to the new http_recv_header() prototype with the done-int-pointer. Signed-off-by: Laslo Hunhold

Refactor http_send_buf() and http_recv_header() and simplify buffer

2020-09-16T08:56:45Z

commit e67eddd517d3f20a24ee14e1eab92806ce5bb20c parent 1a93e3b22f363a0a8dffcf3c47b8d30ff207401f Author: Laslo Hunhold Date: Wed, 16 Sep 2020 10:56:45 +0200 Refactor http_send_buf() and http_recv_header() and simplify buffer The use of "off" was problematic, because it is state that has to be handled carefully and could potentially hit edge-cases (e.g. being larger than sizeof(buf->data)). The real problem is finding a way to indicate function is "done" with something (i.e. done filling or sending the buffer). Instead of playing around with off, which is nothing but a dirty hack, http_send_buf() now just "drains" the buffer, which also makes it idempotent on an empty buffer. It indicates to be "done" as soon as the length of the buffer is zero. The function http_recv_header() gets an additional paramter "done" (an int-pointer) which is used to indicate completeness. As a result, we can drop the off-variable completely from the buffer. Signed-off-by: Laslo Hunhold

Add config.h dependency to quark-rule

2020-09-14T17:50:11Z

commit 1a93e3b22f363a0a8dffcf3c47b8d30ff207401f parent b98cdd453f1c997f241f293f4de8823d54c0d693 Author: Laslo Hunhold Date: Mon, 14 Sep 2020 19:50:11 +0200 Add config.h dependency to quark-rule Due to the refactoring of the header-dependencies it would happen that a compilation unit would pull in config.h before it was created, in case it didn't exist already. Signed-off-by: Laslo Hunhold

Consistently call "enum status"-variables "s"

2020-09-14T13:02:08Z

commit b98cdd453f1c997f241f293f4de8823d54c0d693 parent 2714819dfc639098d0531eb3d4f0f5f23708059a Author: Laslo Hunhold Date: Mon, 14 Sep 2020 15:02:08 +0200 Consistently call "enum status"-variables "s" Signed-off-by: Laslo Hunhold

Make the serving process interruptible

2020-09-14T11:45:24Z

commit 2714819dfc639098d0531eb3d4f0f5f23708059a parent 0823ba4c3e480fb5e2c246b8ac6c4783d866ab87 Author: Laslo Hunhold Date: Mon, 14 Sep 2020 13:45:24 +0200 Make the serving process interruptible Ever since I joined suckless and found out that there had been an (inofficial and cancelled) effort to turn quark into a polling-webserver (instead of a forking-webserver), I was intrigued to pick up the task and make it happen. Back then, my C skills weren't nearly as good, and I had no hopes of making it possible. Now, this commit marks a major step towards this goal. Given the static nature of quark, I wanted to try something out that is not really possible with a "dynamic" server: Making the serving process interruptible in constant memory (except dir-listings of course). This can easily be extended to a polling architecture later on, but it most importantly warrants a non-blocking I/O scheme and makes the server more or less immune to sloth attacks (i.e. clients sending requests very slowly), and provides a more flexible approach to connections. Any thread can pick up a connection and continue work on it, without requiring a separate process for each (which might hit the forking limit at some point). If we hit a point where all connections are busy (due to many sloth attacks), one can apply arbitrary complex logic to "cancel" connections that show malicious behaviour (e.g. taking a long time to send the request header, etc.). The following aspects were added/changed to introduce the interruptibility. - Define a general purpose "buffer" struct with a buffer_appendf() utility function. - Change http_send_header() to http_prepare_header_buf() and separate the sending part into a general-purpose function http_send_buf(). - Modify the data_* functions to be based on a progress and operate on buffers. This way, we can indefinitely "interrupt" request serving and always "pick up" where we left off. - Refactor http_recv_header() to operate on the buffer struct instead of "raw" parameters. - Refactor serve() in main.c accordingly. - Introduce BUFFER_SIZE in config.h, which controls the buffer size each connection has. - Refactor Makefile dependencies and employ strict first-level-header- usage (i.e. we explicitly specify what we use with includes in each compilation unit, so make(1) can figure the dependencies out; most prominently, this moves the arg.h-include into main.c, and requires ifdef-guards for config.h). Signed-off-by: Laslo Hunhold

Add logmsg() and refactor connection handling

2020-08-29T11:02:51Z

commit 0823ba4c3e480fb5e2c246b8ac6c4783d866ab87 parent a36b901d404f4d4268384a379fd040898f78b1b3 Author: Laslo Hunhold Date: Sat, 29 Aug 2020 13:02:51 +0200 Add logmsg() and refactor connection handling Also use compound literals for immediate pointers we don't use later (same as with setsockopt() in 32223c96bdee8f94980d3a1877a643a4d59f897f). Signed-off-by: Laslo Hunhold

Add http_send_body() and data_send_error() and refactor

2020-08-28T22:42:54Z

commit a36b901d404f4d4268384a379fd040898f78b1b3 parent db127723c67534d5693fc033f19c855a403d1447 Author: Laslo Hunhold Date: Sat, 29 Aug 2020 00:42:54 +0200 Add http_send_body() and data_send_error() and refactor This turns the data-functions into the only functions "allowed" to send body-data (called with http_send_body()). The previous (hacky) approach of doing this in http_send_header() is not only out of place, it's an easy source of bugs given, for instance, the sending of body data is not expected with HEAD-requests. Given html_escape() is now only used in data.c, we move it there from util.c and make it a static method again. Signed-off-by: Laslo Hunhold

Rename functions in data.h and adapt ifdef

2020-08-28T21:46:12Z

commit db127723c67534d5693fc033f19c855a403d1447 parent a94b15814cf5729a377dd23c7961f183c6884b59 Author: Laslo Hunhold Date: Fri, 28 Aug 2020 23:46:12 +0200 Rename functions in data.h and adapt ifdef Signed-off-by: Laslo Hunhold

Rename resp.{c,h} to data.{c,h}

2020-08-28T21:29:54Z

commit a94b15814cf5729a377dd23c7961f183c6884b59 parent 9a95d9183c0d4c656d9aca33c2fca2327dc5f3a6 Author: Laslo Hunhold Date: Fri, 28 Aug 2020 23:29:54 +0200 Rename resp.{c,h} to data.{c,h} The methods in data.h only deal with the actual response data, not the request handling itself, which has been formalized a bit more in http.h. To avoid confusion, we rename it to data.h. Signed-off-by: Laslo Hunhold

Rename status to s in serve()

2020-08-28T21:19:29Z

commit 9a95d9183c0d4c656d9aca33c2fca2327dc5f3a6 parent 68e4ff3021d558e1ff3db1767d1b692cbda70c7c Author: Laslo Hunhold Date: Fri, 28 Aug 2020 23:19:29 +0200 Rename status to s in serve() This is more consistent with the codebase. Signed-off-by: Laslo Hunhold

Return proper error-status when http_send_header() fails

2020-08-28T21:16:47Z

commit 68e4ff3021d558e1ff3db1767d1b692cbda70c7c parent c0909c70e4767fa47b44b0964cf03e7962e430c3 Author: Laslo Hunhold Date: Fri, 28 Aug 2020 23:16:47 +0200 Return proper error-status when http_send_header() fails Explicitly show that we set the status of the response struct to the returned error status. This makes it clear that we are beyond the point where the "form" of the response struct matters and it's now only about the log-output. Signed-off-by: Laslo Hunhold

Improve http_prepare_response()'s error semantics

2020-08-28T20:48:32Z

commit c0909c70e4767fa47b44b0964cf03e7962e430c3 parent 123f168a3b5d1e378aa2827d6306a0270b553f90 Author: Laslo Hunhold Date: Fri, 28 Aug 2020 22:48:32 +0200 Improve http_prepare_response()'s error semantics I don't like the juggling with status-values in serve. It makes sense for http_recv_header() and http_parse_header(), because we don't have a response-struct yet that we can "fill". We could pass it to them, but that would make the usage a bit messy. However, in http_prepare_response(), we are already entrusted with a pointer to a response-struct, and just failing here (by returning an error value) leaves the response-struct in an invalid state. Instead, we make it a void function and reflect the status using the status field in the passed response struct. This way, there is no case where the response struct is in an invalid state after calling a http_prepare_*()-method. Signed-off-by: Laslo Hunhold

Replace http_send_status() with http_prepare_error_response()

2020-08-28T20:32:47Z

commit 123f168a3b5d1e378aa2827d6306a0270b553f90 parent 601b56d27095e4340e5afcf3385465a1f7f96d98 Author: Laslo Hunhold Date: Fri, 28 Aug 2020 22:32:47 +0200 Replace http_send_status() with http_prepare_error_response() This approach fits better in line of first initializing the response struct and then sending the header with http_send_header() later. Signed-off-by: Laslo Hunhold

Mention default behaviour in the manual when the host is not given

2020-08-23T11:36:56Z

commit 601b56d27095e4340e5afcf3385465a1f7f96d98 parent 27f8bbfac440d95ef354005d5f7353ec9f3d9294 Author: Laslo Hunhold Date: Sun, 23 Aug 2020 13:36:56 +0200 Mention default behaviour in the manual when the host is not given Signed-off-by: Laslo Hunhold

Refactor sock_get_uds() a bit

2020-08-23T11:35:49Z

commit 27f8bbfac440d95ef354005d5f7353ec9f3d9294 parent 1ccaac023caa4ae415041c190516b0f7ae6ba647 Author: Laslo Hunhold Date: Sun, 23 Aug 2020 13:35:49 +0200 Refactor sock_get_uds() a bit This refines the error messages a bit and makes clearer what went wrong. Signed-off-by: Laslo Hunhold

Rename s to srv

2020-08-23T09:02:38Z

commit 1ccaac023caa4ae415041c190516b0f7ae6ba647 parent 50c85ec642d1327135eb5a58c6d1ffc1ee0d41dc Author: Laslo Hunhold Date: Sun, 23 Aug 2020 11:02:38 +0200 Rename s to srv This improves readability a bit and helps iron out confusions with status-variables called s in other methods. Signed-off-by: Laslo Hunhold

Rename "target" to "URI" where appropriate

2020-08-22T21:37:08Z

commit 50c85ec642d1327135eb5a58c6d1ffc1ee0d41dc parent 68be64e2c12f6ab5355a147484896eae12d2b166 Author: Laslo Hunhold Date: Sat, 22 Aug 2020 23:37:08 +0200 Rename "target" to "URI" where appropriate Of course URIs point at "targets", but the URIs themselves should be called what they are, not only in the interest of clarity in terms of nomenclature. Signed-off-by: Laslo Hunhold

Remove unused field in the request-struct

2020-08-22T21:31:32Z

commit 68be64e2c12f6ab5355a147484896eae12d2b166 parent 58d0f44e0395fe37b3575da35992b3d3e7f262d7 Author: Laslo Hunhold Date: Sat, 22 Aug 2020 23:31:32 +0200 Remove unused field in the request-struct Signed-off-by: Laslo Hunhold

Refactor http_send_response() into http_prepare_response()

2020-08-22T21:20:00Z

commit 58d0f44e0395fe37b3575da35992b3d3e7f262d7 parent a5163d08135b81b271b1d7ba36b24e342b57961f Author: Laslo Hunhold Date: Sat, 22 Aug 2020 23:20:00 +0200 Refactor http_send_response() into http_prepare_response() The function http_send_response() did too much. It not only took the request fields and built them together into a response, it delegated too little and many functions were "hacked" into it, for instance shady directory-changes for vhosts and hand-construction of response structs. The preparations for a rework were already made in previous commits, including a tighter focus on the response-struct itself. Instead of doing everything locally in the http_send_response() function, the new http_prepare_response() only really takes the request-struct and builds a response-struct. The response-struct is expanded such that it's possible to do the data-sending simply with the response-struct itself and not any other magic parameters that just drop out of the function. Another matter are the http_send_status()-calls. Because the aforementioned function is so central, this refactoring has included many areas. Instead of calling http_send_status() in every error-case, which makes little sense now given we first delegate everything through a response struct, errors are just sent as a return value and caught centrally (in serve() in main.c), which centralizes the error handling a bit. It might look a bit strange now and it might not be clear in which direction this is going, but subsequent commits will hopefully give clarity in this regard. Signed-off-by: Laslo Hunhold

Split up http_get_request()

2020-08-22T09:05:20Z

commit a5163d08135b81b271b1d7ba36b24e342b57961f parent c1b242e405d40067c282e8116d21c6f2641e4eee Author: Laslo Hunhold Date: Sat, 22 Aug 2020 11:05:20 +0200 Split up http_get_request() The function has become too long and basically did two things: Receiving the header and parsing it. To better reflect this, we split it up into the two functions http_recv_header() and http_parse_header(). This way, we also obtain a better separation of concerns and can further reduce the scope of each parameter-list. http_recv_header() has been written in such a way that it can be reentered and fill up the header-buffer bit by bit using a pointer to an offset value. The error handling was improved by only returning the immediate error status codes and letting the caller do the error-handling with http_send_status(). Signed-off-by: Laslo Hunhold

Add connection struct

2020-08-22T07:24:57Z

commit c1b242e405d40067c282e8116d21c6f2641e4eee parent 6d2fe7f29e12190d9be062852cf3f21b7f695369 Author: Laslo Hunhold Date: Sat, 22 Aug 2020 09:24:57 +0200 Add connection struct This struct contains the request and response structs, represents a state and has some utility-buffers. Signed-off-by: Laslo Hunhold

Move infd and header into request-struct

2020-08-21T17:38:29Z

commit 6d2fe7f29e12190d9be062852cf3f21b7f695369 parent ce77dd79624177387dbfc95cabae7c851b51b7fb Author: Laslo Hunhold Date: Fri, 21 Aug 2020 19:38:29 +0200 Move infd and header into request-struct This compacts the connection state into one struct. Signed-off-by: Laslo Hunhold

Update manpage to list capabilities and behaviour

2020-08-18T06:46:52Z

commit ce77dd79624177387dbfc95cabae7c851b51b7fb parent 65600ffe7a2868e95cf172550c85aa074e209e0d Author: Laslo Hunhold Date: Tue, 18 Aug 2020 08:46:52 +0200 Update manpage to list capabilities and behaviour Signed-off-by: Laslo Hunhold

Reduce global state by localizing the server-struct

2020-08-17T09:37:25Z

commit 65600ffe7a2868e95cf172550c85aa074e209e0d parent 3bd49b24561ce3c7be916ab0abbc78288721ddc4 Author: Laslo Hunhold Date: Mon, 17 Aug 2020 11:37:25 +0200 Reduce global state by localizing the server-struct The server-struct variable s was global, which made it readable and modifiable from any point in the code. Making it a local variable in main() instead and passing it as a pointer to constant memory to each function needing it makes much more sense and allows the compiler to warn us if we do try to modify it, which it wouldn't have before. Signed-off-by: Laslo Hunhold

Implement RFC 8615 (Well-Known URIs) and refine access errors

2020-08-17T08:33:55Z

commit 3bd49b24561ce3c7be916ab0abbc78288721ddc4 parent 660699492fb2275335d24fb26ec3c6529f623af0 Author: Laslo Hunhold Date: Mon, 17 Aug 2020 10:33:55 +0200 Implement RFC 8615 (Well-Known URIs) and refine access errors We generally rejected any URI that had a path component beginning with a '.', i.e. a hidden file. RFC 8615 specifies the well-known URI, which is used, for instance, with the "http-01" challenge type in acme-client(1) and will probably see more usage in the future. To support it, we move the hidden target check after the stat(), so we don't have to worry about canonicalization of dir-URIs (i.e. missing trailing '/'). This changes the behaviour a bit, as now quark won't only send out a 403 whenever a hidden target is requested, but only if it actually exists, and a 404 otherwise. Given the earlier call to normabspath() ensures that our path begins with a '/', we don't need the first check "realtarget[0] == '.'" anymore, so it can be removed. Thanks to Robert Russell for reporting the lack of support of the RFC 8615 in quark. Signed-off-by: Laslo Hunhold

Make user/group-handling-code more robust

2020-08-09T21:20:06Z

commit 660699492fb2275335d24fb26ec3c6529f623af0 parent b1dca4cf97d49750134fd98f0824f7913361c9c5 Author: Laslo Hunhold Date: Sun, 9 Aug 2020 23:20:06 +0200 Make user/group-handling-code more robust As is there is no security issue, but _if_ we end up with a user or group set to NULL after e.g. ARGEND, we would've hit a null-pointer- dereference of grp in which is now line 311. What we want to check instead is if user or group are NULL respectively and throw an error. Consequently, we can remove the later checks in the drop root section, as we now guarantee that grp and pwd are not NULL. Signed-off-by: Laslo Hunhold

Remove three dead stores in main()

2020-08-09T20:43:46Z

commit b1dca4cf97d49750134fd98f0824f7913361c9c5 parent 03ee1df4c30bb6d6359381ebb7977ac08cd6b5dc Author: Laslo Hunhold Date: Sun, 9 Aug 2020 22:43:46 +0200 Remove three dead stores in main() Signed-off-by: Laslo Hunhold

Add space in list

2020-08-05T21:27:05Z

commit 03ee1df4c30bb6d6359381ebb7977ac08cd6b5dc parent 2318a89ecd7aad5a296b657caec22beff92a4284 Author: Laslo Hunhold Date: Wed, 5 Aug 2020 23:27:05 +0200 Add space in list Thanks Hiltjo! Signed-off-by: Laslo Hunhold

Begin comment in lowercase

2020-08-05T17:14:10Z

commit 2318a89ecd7aad5a296b657caec22beff92a4284 parent cb7a1f6390094a9fc84376d4c6c6eb6e0f2ddf0b Author: Laslo Hunhold Date: Wed, 5 Aug 2020 19:14:10 +0200 Begin comment in lowercase Signed-off-by: Laslo Hunhold

Replace off_t with size_t

2020-08-05T16:59:55Z

commit cb7a1f6390094a9fc84376d4c6c6eb6e0f2ddf0b parent d105c28aad2b90955d7cbbaabd27ff4193aff686 Author: Laslo Hunhold Date: Wed, 5 Aug 2020 18:59:55 +0200 Replace off_t with size_t While off_t might be better suited for file-offsets and -sizes, the IEEE Computer Society was unable to mandate limits (min, max) for it in the POSIX specification in the last 32 years. Because it's impossible to portably determine these numbers for signed integers, I decided to switch to size_t for the offsets to be able to pass proper values to strtonum(), because C99 is sane and has defined limits for size_t (i.e. SIZE_MIN and SIZE_MAX). On my system, long long and off_t have the same size, so it didn't trigger any bugs, but strtonum() could pass a bigger number to lower and upper than they can handle and make them overflow. The rationale for switching to size_t is actually given by the fact that functions like mmap() blur the border between memory and filesystem. Another point is that glibc has a horrible define _FILE_OFFSET_BITS you need to set to 64 to actually get decent values for off_t, which was a huge headache in sbase until we found that out. Signed-off-by: Laslo Hunhold

Ensure const-correctness where possible and refactor parse_range()

2020-08-05T16:28:21Z

commit d105c28aad2b90955d7cbbaabd27ff4193aff686 parent 90d5179ea0d7a437a08085dfa1c10953dbd45a68 Author: Laslo Hunhold Date: Wed, 5 Aug 2020 18:28:21 +0200 Ensure const-correctness where possible and refactor parse_range() I know that the effect of 'const' on compiler optimizations is smaller than many believe, but it provides a good insight to the caller which parameters are not modified and simplifies parallelization, in case that is desired at a later point. Throughout processing, the big structs mostly remained unmodified, with the exception of parse_range(), which added a null-byte in the "Range"- header to simplify its parsing. This commit refactors parse_range() such that it won't modify this string anymore. Additionally, the parser was made even stricter: Usually, strtoll() (which is wrapped by strtonum()) allows whitespace and plus and minus signs before the number, which is not part of the specification. The stricter parser also better differentiates now between invalid requests and range-lists. In that context, the switch in http_send_response() was replaced for better readability. Signed-off-by: Laslo Hunhold

Rename REQ_MOD to REQ_IF_MODIFIED_SINCE

2020-08-05T13:46:03Z

commit 90d5179ea0d7a437a08085dfa1c10953dbd45a68 parent 2c50d0c654ca619fc3c3e34ee3452ffdf9ef7c4e Author: Laslo Hunhold Date: Wed, 5 Aug 2020 15:46:03 +0200 Rename REQ_MOD to REQ_IF_MODIFIED_SINCE The named constants for header fields of the response struct all pretty much matched the actual header name, which I think improves readability for everyone familiar with the HTTP-spec. The request header fields named constants followed the rule, except the "If-Modified-Since"-header, which is addressed in this commit. Signed-off-by: Laslo Hunhold

Rename request "r" to "req"

2020-08-05T13:43:29Z

commit 2c50d0c654ca619fc3c3e34ee3452ffdf9ef7c4e parent c51b31d7ac2c8bb8eeb6cb058c2d5d95fc4a52b8 Author: Laslo Hunhold Date: Wed, 5 Aug 2020 15:43:29 +0200 Rename request "r" to "req" Now that we have response-structs called "res", the naming "r" is a bit ambiguous. Signed-off-by: Laslo Hunhold

Refactor response-generation

2020-08-05T11:41:44Z

commit c51b31d7ac2c8bb8eeb6cb058c2d5d95fc4a52b8 parent 26c593ade1949608a4d5a8a1b70eb6b11d8dc316 Author: Laslo Hunhold Date: Wed, 5 Aug 2020 13:41:44 +0200 Refactor response-generation I wasn't happy with how responses were generated. HTTP-headers were handled by hand and it was duplicated in multiple parts of the code. Due to the duplication, some functions like timestamp() had really ugly semantics. The HTTP requests are parsed much better: We have an enum of fields we care about that are automatically read into our request struct. This commit adapts this idea to the response: We have an enum of fields we might put into our response, and a response-struct holds the content of these fields. A function http_send_header() automatically sends a header based on the entries in response. In case we don't use a field, we just leave the field in the response-struct empty. With this commit, some logical changes came with it: - timestamp() now has a sane signature, TIMESTAMP_LEN is no more and it can now return proper errors and is also reentrant by using gmtime_r() instead of gmtime() - No more use of a static timestamp-array, making all the methods also reentrant - Better internal-error-reporting: Because the fields are filled before and not during sending the response-headers, we can better report any internal errors as status 500 instead of sending a partial non-500-header and then dying. These improved data structures make it easier to read and hack the code and implement new features, if desired. Signed-off-by: Laslo Hunhold

Refactor range-parsing into a separate function

2020-08-04T14:31:08Z

commit 26c593ade1949608a4d5a8a1b70eb6b11d8dc316 parent 5a7994bc61ce868521fc996e46006b61d07d23c4 Author: Laslo Hunhold Date: Tue, 4 Aug 2020 16:31:08 +0200 Refactor range-parsing into a separate function The method http_send_response() is already long enough and this separation of concerns both helps shorten it a bit, improves readability and reduces the chance of programming errors. Signed-off-by: Laslo Hunhold

Send Accept-Ranges-header for file-requests

2020-07-23T16:54:43Z

commit 5a7994bc61ce868521fc996e46006b61d07d23c4 parent db4e35d3d5cbd5bfac61cfb8edadd47c4608a864 Author: Laslo Hunhold Date: Thu, 23 Jul 2020 18:54:43 +0200 Send Accept-Ranges-header for file-requests Now that the range-support is actually working, we can send out the Accept-Ranges-header so that clients know they can send range-requests to the server. This can be seen empirically when watching a video and skipping around into unbuffered space. Previously, it would not be possible and the time-selector would flip back to the furthest point the previous buffering had progressed. Now it is working flawlessly. Signed-off-by: Laslo Hunhold

Refactor range-parsing

2020-07-23T16:16:08Z

commit db4e35d3d5cbd5bfac61cfb8edadd47c4608a864 parent 6b508a0e073fd4e29540f8808d919ff72c1893fb Author: Laslo Hunhold Date: Thu, 23 Jul 2020 18:16:08 +0200 Refactor range-parsing Quark previously didn't really handle suffix-range-requests (those of the form "-num", asking for the last num bytes) properly and also did not catch the error when the lower in the range "lower-upper" was actually larger than or equal to the size of the requested file. I always planned to refactor the parsing but got the motivation by Eric Radman , who kindly reported the latter bug to me. Signed-off-by: Laslo Hunhold

Explicitly initialize struct tm with zeroes

2020-07-23T14:54:21Z

commit 6b508a0e073fd4e29540f8808d919ff72c1893fb parent 660b3086172c653fa65b1e2bddd3ce99863f30d9 Author: Laslo Hunhold Date: Thu, 23 Jul 2020 16:54:21 +0200 Explicitly initialize struct tm with zeroes This is recommended by the manual as strptime(), in principle, might only touch the fields it parses from the string. Given the struct tm implementations differ from operating system to operating system, we make sure and set everything to zero before passing it to strptime(). Signed-off-by: Laslo Hunhold

Use timegm() instead of mktime() to generate UNIX-timestamp

2020-07-23T14:48:34Z

commit 660b3086172c653fa65b1e2bddd3ce99863f30d9 parent a55df3915dad471ee0629262e950aefc43b8dbad Author: Laslo Hunhold Date: Thu, 23 Jul 2020 16:48:34 +0200 Use timegm() instead of mktime() to generate UNIX-timestamp The broken down time-representation tm generated earlier is in UTC, and mktime() assumes that it's in local time instead, leading to the problem that quark might not send a NOT_MODIFIED in a different timezone. timegm() instead correctly interprets the broken down time-representation tm as UTC and returns the proper timestamp. It might not be portable like mktime(), but it's complicated to emulate it otherwise. Thanks to Jeremy Bobbin for reporting the bug and providing this fix, which is why I've added him to the LICENSE. Thanks also to Hiltjo for his input. Signed-off-by: Laslo Hunhold

Update LICENSE

2020-05-07T11:41:11Z

commit a55df3915dad471ee0629262e950aefc43b8dbad parent b7d0d6889df57e6d288ab4b3ddc1a243558bfa9d Author: Laslo Hunhold Date: Thu, 7 May 2020 13:41:11 +0200 Update LICENSE Signed-off-by: Laslo Hunhold

Fix for sending HTTP response status 304

2020-04-22T18:46:30Z

commit b7d0d6889df57e6d288ab4b3ddc1a243558bfa9d parent 9dda0028db2dce035e3dab84b4376c25b21302fa Author: Rainer Holzner Date: Wed, 22 Apr 2020 20:46:30 +0200 Fix for sending HTTP response status 304 Stop immediately after responding with status code 304 "Not Modified". This also solves missing log output for status 304. If there is an error while sending a file, try to clean up and close the file.

Update LICENSE

2020-04-21T15:48:20Z

commit 9dda0028db2dce035e3dab84b4376c25b21302fa parent 8ccef4b27a72f339b820f91c82330a3a5b592f7b Author: Laslo Hunhold Date: Tue, 21 Apr 2020 17:48:20 +0200 Update LICENSE Signed-off-by: Laslo Hunhold

Make host parameters optional

2020-04-21T15:04:37Z

commit 8ccef4b27a72f339b820f91c82330a3a5b592f7b parent 48e74a598247f4b81e09a0f652faf15163f9f525 Author: Nihal Jere Date: Tue, 21 Apr 2020 17:04:37 +0200 Make host parameters optional For me at least, the first valid configuration found by getaddrinfo works fine most of the time. Obviously if this isn't the configuration you want, you can specify the host explicitly.

Properly HTML-escape names in dirlistings

2020-03-25T13:07:17Z

commit 48e74a598247f4b81e09a0f652faf15163f9f525 parent 5ee8c07e7e3e601fce49fbc2b170227924be3804 Author: Laslo Hunhold Date: Wed, 25 Mar 2020 14:07:17 +0100 Properly HTML-escape names in dirlistings Based on a patch by guysv. We now make sure that the valid path-characters ", ', <, >, & can not be used for XSS on a target, for example with a file called ">

Fix unveil(2) usage

2020-03-20T19:35:34Z

commit 5ee8c07e7e3e601fce49fbc2b170227924be3804 parent 3c7049e9063edebbd1934178f263f9f3c9b8ddf5 Author: Laslo Hunhold Date: Fri, 20 Mar 2020 20:35:34 +0100 Fix unveil(2) usage Thanks to the feedback by z0lqLA! I forgot that unveil(NULL, NULL) only locks further unveil calls when there has been at least _one_ prior call to unveil! To fix this, we reorder the calls and also make sure to call unveil() before we disallow unveils via pledge. Signed-off-by: Laslo Hunhold

Use pledge(2) and unveil(2) on OpenBSD

2019-09-23T14:56:28Z

commit 3c7049e9063edebbd1934178f263f9f3c9b8ddf5 parent 32223c96bdee8f94980d3a1877a643a4d59f897f Author: Laslo Hunhold Date: Mon, 23 Sep 2019 16:56:28 +0200 Use pledge(2) and unveil(2) on OpenBSD It has been on my todo-list for a long time. I tested it on OpenBSD 6.5. Thanks Richard Ulmer for the reminder. Signed-off-by: Laslo Hunhold

Use compound literals and explicit initialization

2019-05-30T21:15:47Z

commit 32223c96bdee8f94980d3a1877a643a4d59f897f parent 33def953e9d216036ad921ac26052a5b0fa790dc Author: Laslo Hunhold Date: Thu, 30 May 2019 23:15:47 +0200 Use compound literals and explicit initialization I didn't really like the use of a "yes"-variable for setsockopt(). A better way is to use compound literals (part of C99). Another point are the structs. Instead of memsetting to zero we make use of the standard which guarantees that "unmentioned" fields are set to zero anyways. Just to note it here: The use of memset() also sets padding to zero, which is not guaranteed with the method of "unmentioned" fields. Signed-off-by: Laslo Hunhold

Improve tokenization for m- and v-flag parsing

2019-02-24T20:50:39Z

commit 33def953e9d216036ad921ac26052a5b0fa790dc parent 065394cb64395ed9a6e7cc57e2519ee8f754ce3e Author: Laslo Hunhold Date: Sun, 24 Feb 2019 21:50:39 +0100 Improve tokenization for m- and v-flag parsing I wasn't happy with the tokenizer for the m- and v-flags, because it was handling space-separated input and there was no way to have spaces within the tokens themselves. This is a fine detail, but I didn't want to impose this restriction where it could be solved (path prefixes or folder names can very well contain spaces). Given it's a bit quirky to handle multiple arguments to a single flag in the command line, especially when parameters are optional, this alternative wasn't further considered and I instead implemented a tokenizer that allows escaping spaces with '\'. While at it, I clarified the manual regarding this point. Signed-off-by: Laslo Hunhold

Change target prefix mapping argument order

2019-02-23T23:40:46Z

commit 065394cb64395ed9a6e7cc57e2519ee8f754ce3e parent 48ddb8fefb512effefd9b761efa25fd9158e2edc Author: Laslo Hunhold Date: Sun, 24 Feb 2019 00:40:46 +0100 Change target prefix mapping argument order Put the chost-specification at the end and make it optional. This makes more sense than having to give an arbitrary useless name in case you weren't using virtual hosts in the first place. While at it, clear up the wording in the manpage. Signed-off-by: Laslo Hunhold

Sort flag-switch alphabetically

2019-02-23T12:50:59Z

commit 48ddb8fefb512effefd9b761efa25fd9158e2edc parent f2afbc4dd703e07b6204faa6d29f885482b18a3b Author: Laslo Hunhold Date: Sat, 23 Feb 2019 13:50:59 +0100 Sort flag-switch alphabetically Signed-off-by: Laslo Hunhold

Add a space after the number in the Xr mandoc macro

2019-02-18T22:44:12Z

commit f2afbc4dd703e07b6204faa6d29f885482b18a3b parent e299e186edba03192fc12f6709df48d02aa83849 Author: Laslo Hunhold Date: Mon, 18 Feb 2019 23:44:12 +0100 Add a space after the number in the Xr mandoc macro Detected with the mandoc(1)-linter. Signed-off-by: Laslo Hunhold

Don't replace '+' with ' ' when decoding URLs

2019-01-10T21:02:23Z

commit e299e186edba03192fc12f6709df48d02aa83849 parent bbd47e1427940e0d4f22a098acd593c1365accd3 Author: Laslo Hunhold Date: Thu, 10 Jan 2019 22:02:23 +0100 Don't replace '+' with ' ' when decoding URLs After the initial report by Platon Ryzhikov, I couldn't validate this behaviour with the given RFC 3986[0], which only speaks of percent encoding for reserved characters. [0]:https://tools.ietf.org/html/rfc3986 Signed-off-by: Laslo Hunhold

Specify UTF-8 for non-binary content-types

2019-01-02T16:04:23Z

commit bbd47e1427940e0d4f22a098acd593c1365accd3 parent d2013a6337972c62a71f01324e87af0e55579245 Author: Laslo Hunhold Date: Wed, 2 Jan 2019 17:04:23 +0100 Specify UTF-8 for non-binary content-types If charset is unspecified, the encoding falls back to ISO 8859-1 or something else that is defined in HTTP/1.1. Given there is no reason not to use UTF-8 nowadays[0] and one can convert legacy encodings to UTF-8 easily, if the case comes up, it is a sane default to specify it in the config.def.h. [0]: https://utf8everywhere.org/ Signed-off-by: Laslo Hunhold

Fix one byte NULL stack overflow

2018-07-16T20:46:09Z

commit d2013a6337972c62a71f01324e87af0e55579245 parent 72b309bbe40444add563cdc37c0aa31386a4630d Author: Aaron Burrow Date: Mon, 16 Jul 2018 22:46:09 +0200 Fix one byte NULL stack overflow Don't append a forward slash if the length of a folder is PATH_MAX-1. This can happen if HEADER_MAX is larger than PATH_MAX or if the `-m` option is used to increase the path length.

Correct arg.h license

2018-07-16T09:47:47Z

commit 72b309bbe40444add563cdc37c0aa31386a4630d parent 9ff3f780e1d1912b89727140df7c1529ab8c5146 Author: Laslo Hunhold Date: Mon, 16 Jul 2018 11:47:47 +0200 Correct arg.h license Credit where credit is due.

Send a relative redirection header wherever possible

2018-07-02T16:43:06Z

commit 9ff3f780e1d1912b89727140df7c1529ab8c5146 parent 34189e0a1f288e05ce528bcab28b922d53d5e471 Author: Laslo Hunhold Date: Mon, 2 Jul 2018 18:43:06 +0200 Send a relative redirection header wherever possible This makes quark much more flexible when it is run behind a network filter or other kind of tunnel. Only send an absolute redirection when we are handling vhosts.

Use sizeof() - 1 rather than strlen()

2018-07-02T16:41:29Z

commit 34189e0a1f288e05ce528bcab28b922d53d5e471 parent b354ffb2385a0bbb8141ec1234b0e1d5874f8031 Author: Laslo Hunhold Date: Mon, 2 Jul 2018 18:41:29 +0200 Use sizeof() - 1 rather than strlen() I know, most compiler probably optimize this anyway, but why not do it right in the first place?

Add Dominik Schmidt to license

2018-07-02T05:15:19Z

commit b354ffb2385a0bbb8141ec1234b0e1d5874f8031 parent 094c8ba814f388dcefee403c4d369005a8d9810f Author: Laslo Hunhold Date: Mon, 2 Jul 2018 07:15:19 +0200 Add Dominik Schmidt to license

Open a new process group before setting up signal handler

2018-07-02T02:08:08Z

commit 094c8ba814f388dcefee403c4d369005a8d9810f parent ba38b0969f542e6e7d3f01ba500189a81ca32355 Author: Dominik Schmidt Date: Mon, 2 Jul 2018 02:08:08 +0000 Open a new process group before setting up signal handler When cleaning up after a caught signal, quark forwards the signal to all processes in the process group with `kill(0, ...)`. If we do not open up a new process group in the parent process, quarks parent will be sent a SIG... too, resulting it to shut down (especially considering that the parent process might run as root). As a result, if we set up the service with djb's excellent daemontools, `svc -d quark` will terminate the svscan-process and tear all other services down with it. See also .

Give an indication of the time zone in the log

2018-04-02T23:23:00Z

commit ba38b0969f542e6e7d3f01ba500189a81ca32355 parent 3ff82c514becd08922fcf9bc9f4870941650932a Author: Laslo Hunhold Date: Tue, 3 Apr 2018 01:23:00 +0200 Give an indication of the time zone in the log We use Zulu-time (aka UTC) for the log timestamps.

Clean up request host properly

2018-04-02T22:55:52Z

commit 3ff82c514becd08922fcf9bc9f4870941650932a parent c3ddb2dd14bd7a39dedbbf3520c9a2052dd3e1ff Author: Laslo Hunhold Date: Tue, 3 Apr 2018 00:55:52 +0200 Clean up request host properly We all agree that the IPv6 address format is a big clusterfuck and only an insane person would've come up with it given the double colons interfere with the way one actually appends a port to a normal IPv4 address. To counteract in this issue, the RFC specifies that one should enclose IPv6-addresses in square brackets to make the disctinction possible, i.e. host: ::1 port: 80 --> [::1]:80 The host field can contain both a port suffix and, of course by the RFC, have the address enclosed in square brackets. Given I personally see this as a "transport enclosure" I'd rather like to see it gone as soon as possible and thus implement this cleanup in the http-header-parser so the output is nice and clean and we don't have to deal with this garbage later on. Thanks to Josuah Demangeon for his wonderful input and his dedication to read the RFCs 3986 and 2732 in such great detail.

permit prefix to be empty in -v format string

2018-04-02T00:55:00Z

commit c3ddb2dd14bd7a39dedbbf3520c9a2052dd3e1ff parent 69bb7710eb9bcc73dd0836b9b88228259c9586d3 Author: Josuah Demangeon Date: Mon, 2 Apr 2018 02:55:00 +0200 permit prefix to be empty in -v format string The previous parsing of the -v vhosts made sure there were 4 tokens. If there was no prefix specified, usage() is called. Now, it only checks for the firsts 3, with .prefix set to null if there are only 3 tokens.

fix segfault on parsing of -v and -m

2018-04-02T00:54:59Z

commit 69bb7710eb9bcc73dd0836b9b88228259c9586d3 parent 6770dc06e69f422124d1e9c0efdefa7cc4e98b61 Author: Josuah Demangeon Date: Mon, 2 Apr 2018 02:54:59 +0200 fix segfault on parsing of -v and -m The length is initially 0 so it needs to be incremented before reallocarray to avoid ...alloc(0); and keep some space for the element to insert.

Add netinet/in.h to sock.c

2018-03-05T09:24:46Z

commit 6770dc06e69f422124d1e9c0efdefa7cc4e98b61 parent a20136fa18dba7fd44812be235010a4517aef783 Author: Laslo Hunhold Date: Mon, 5 Mar 2018 10:24:46 +0100 Add netinet/in.h to sock.c It was missing but necessary for some defines.

Update the documentation to reflect the new flag-centric usage

2018-03-05T08:51:29Z

commit a20136fa18dba7fd44812be235010a4517aef783 parent 444b8f5b32d0263f1a20e18eb3044bfeed334361 Author: Laslo Hunhold Date: Mon, 5 Mar 2018 09:51:29 +0100 Update the documentation to reflect the new flag-centric usage

http_send_response: fix undefined behaviour for copying the target string

2018-03-05T00:12:09Z

commit 444b8f5b32d0263f1a20e18eb3044bfeed334361 parent ed8b7e8954d302f73907f1cc302d124443f947aa Author: Hiltjo Posthuma Date: Mon, 5 Mar 2018 01:12:09 +0100 http_send_response: fix undefined behaviour for copying the target string ... the format string and buffer were the same (undefined behaviour).

Fix a logic error

2018-03-05T00:04:51Z

commit ed8b7e8954d302f73907f1cc302d124443f947aa parent 01ed0dac83ed27faf5e8dc90c893b1cd0e8a917f Author: Laslo Hunhold Date: Mon, 5 Mar 2018 01:04:51 +0100 Fix a logic error We want to xor s.host and udsname, so it especially errors out when none are given.

util: don't initialize server, it's already done in main()

2018-03-04T23:56:24Z

commit 01ed0dac83ed27faf5e8dc90c893b1cd0e8a917f parent c8401c591fac7fd98349e05e595cdbe861998a90 Author: Hiltjo Posthuma Date: Mon, 5 Mar 2018 00:56:24 +0100 util: don't initialize server, it's already done in main() ... this removes a stupid compiler warning too.

Add esnprintf() and refactor some code

2018-03-04T23:59:37Z

commit c8401c591fac7fd98349e05e595cdbe861998a90 parent 1879e14e79aca6f676d48e58500eb755f341e78b Author: Laslo Hunhold Date: Mon, 5 Mar 2018 00:59:37 +0100 Add esnprintf() and refactor some code The (size_t) discards the case where the return value of snprintf is < 0. This is rather unlikely, but we'll keep it in mind anyway.

Be extra pedantic again and remove all warnings

2018-03-04T23:30:53Z

commit 1879e14e79aca6f676d48e58500eb755f341e78b parent 3ff3e5ea6e1a3d85c6282a833a07424fcac3acc3 Author: Laslo Hunhold Date: Mon, 5 Mar 2018 00:30:53 +0100 Be extra pedantic again and remove all warnings Since now config.def.h has been reduced we don't have any more unused variables and thus the manual fiddling with error-levels is no longer necessary. To get a completely clean result though we have to still cast some variables here and there.

Add some missing headers and interface visibility macro

2018-03-01T14:18:20Z

commit 3ff3e5ea6e1a3d85c6282a833a07424fcac3acc3 parent 6b55e36036d52c07803824048080a1e350fe6c8c Author: Quentin Rameau Date: Thu, 1 Mar 2018 15:18:20 +0100 Add some missing headers and interface visibility macro strings.h for strncasecmp time.h for strptime