Feature #3264
closed[PATCH] sort by the exact value of mtime or size
Description
This change implements directory list sorting by file modification time or size, with the exact raw values instead of the 'human-readable' strings. It is implemented by attaching the raw value into the 'td' element; which looks like:
<td class="n"><a href="config.log">config.log</a></td><td class="m" data-value="1725886319">2024-Sep-09 20:51:59</td><td class="s" data-value="142655">139.3K</td><td class="t">text/plain</td>
This provides multiple benefits. For example, when sorting my mtime, the human-readable date time string is longer used for sorting, enabling (future) customization of the date time string; and for sorting by size, files with similar sizes can now be sorted correctly; the sorting code no longer have to scale the human-readable size string back to an imprecise number, as it was done by function 'unit_multiplier'.
Files
Updated by gstrauss 3 months ago · Edited
Thank you. I will review further later this week.
For others reading this:
HTML data attributes are reasonably well-supported across modern clients which support JavaScript:
https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/dataset
https://developer.mozilla.org/en-US/docs/Learn/HTML/Howto/Use_data_attributes
Updated by gstrauss 3 months ago
This provides multiple benefits. For example, when sorting my mtime, the human-readable date time string is longer used for sorting, enabling (future) customization of the date time string; and for sorting by size, files with similar sizes can now be sorted correctly; the sorting code no longer have to scale the human-readable size string back to an imprecise number, as it was done by function 'unit_multiplier'.
This patch increases the size of the HTML due to the additional data-value entries, and marginally increases processing time on the server to produce the extra data-value entries. The benefits of this patch are largely a more efficient sort of modification time, should the user undertake to do so. As you noted, sorting by size is also more precise where before there could be inaccuracies in sorting.
If sorting by modification time or by size, then this patch has clear benefits. If not doing so, there are additional costs incurred by every page served by mod_dirlisting.
enabling (future) customization of the date time string
What do you have in mind? Should this sort of thing be done in javascript on the client, and should the client make a request to lighttpd for mod_dirlisting JSON output?
Updated by Low-power 3 months ago · Edited
gstrauss wrote in #note-3:
What do you have in mind? Should this sort of thing be done in javascript on the client, and should the client make a request to lighttpd for mod_dirlisting JSON output?
I think comparing to server-side sorting, like the one implemented in Apache HTTP server, sorting in client-side provides one big advantage is that it will usually be done very quickly, since the browser don't need to send another request to server, and wait for it. Therefore sending separated requests for the JSON output may break this good experience.
For the motivation this patch, I thought since lighttpd has the resorting implemented in client-side, it should be done correctly and precisely (for the size column). With sorting by exact size value implemented in my local lighttpd build, I looked into the timestamp column and decide to do a similar modification on that too. While I did modified the date time format for my server setup after that, it is not the initial motivation for this modification.
gstrauss wrote in #note-2:
Is this patch addition necessary?
[...]
This provides a failsafe in case the data-value is missing from the mtime td element. Could be unnecessary indeed, considering a missing data-value an exceptional condition for the mtime column.
Edit:
Just realized this failsafe was implemented for the '../' entry. It is hard-coded to have no data-values, so the sorting code will need to treat it to have a mtime of 1970-01-01 00:00:00.
Updated by gstrauss 3 months ago
Just realized this failsafe was implemented for the '../' entry. It is hard-coded to have no data-values, so the sorting code will need to treat it to have a mtime of 1970-01-01 00:00:00.
Thanks for noting that.
Instead, maybe we should hard-code data-value="0" for '../' ?
Updated by gstrauss 3 months ago
I think comparing to server-side sorting, like the one implemented in Apache HTTP server, sorting in client-side provides one big advantage is that it will usually be done very quickly, since the browser don't need to send another request to server, and wait for it. Therefore sending separated requests for the JSON output may break this good experience.
FYI: Using JSON allows for a streamed response since sorting is not done on the server side. This may result in lower resource usage on the server, often a good thing. For larger directory listings, this can also actually improve the client experience since the client has been receiving data as lighttpd reads it from disk and the client can choose how/when to display the received data, rather than having to wait for lighttpd to read the entire directory and sort it before starting to send a response.
Note: lighttpd sorts the directory listing by name by default, for the benefit of display to clients without (or having disabled) javascript. This can be disable with dir-listing.sort = "disable"
Updated by Low-power 3 months ago
For the optimizating size of the resulting HTML, I think it should support an option to completely disable client-side sorting; this way, all the data values, as well as the JS codes for sorting, can be eliminated. The final HTML size will be greatly reduced, for the potential extreme use cases.
Updated by gstrauss 3 months ago
For the optimizating size of the resulting HTML, I think it should support an option to completely disable client-side sorting; this way, all the data values, as well as the JS codes for sorting, can be eliminated. The final HTML size will be greatly reduced, for the potential extreme use cases.
There are many, many approaches to reducing the size of the HTML. One more extreme method would be producing text, not HTML, so at the moment, I do not plan to special case omitting data-value and inline javascript.
I have a tentative patch, based on yours but further modified, on my dev branch
https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/84cc83af0d9f1a8d5c3317d8c87e06ae1b19dc24
Updated by gstrauss 3 months ago
I will likely split the commit on my dev branch into two parts, separating the changes to mtime to a separate commit, and I may revert that separate commit. The date is a large integer and data-value="large-integer" unnecessarily increases the size of the resulting document. While sorting mtime is faster with the data-value, there is a measurable cost in document size. If the mtime data-value is reverted, for short directory listings, the slower date parsing is not a big deal; for larger directory listings, the slower date parsing cost is incurred only if sorting by mtime, and could be made faster by parsing mtime and saving into data-value for subsequent comparisons.
The additional document size due to storing actual size in data-value is acceptable since it is more precise and fixes existing inaccuracy of sorting files of approx the same size.
Updated by gstrauss 3 months ago
- Status changed from New to Patch Pending
My patches no longer do the following (commented out) to generate data-value for mtime:
@@ -990,6 +994,11 @@ static void http_list_directory_header(request_st * const r, const handler_ctx *
}
static void http_list_directory_mtime(buffer * const out, const dirls_entry_t * const ent) {
+#if 0
+ char n[LI_ITOSTRING_LENGTH];
+ buffer_append_string_len(out, n, li_itostrn(n, sizeof(n), TIME64_CAST(ent->mtime)));
+ buffer_append_string_len(out, CONST_STR_LEN("\">"));
+#endif
struct tm tm;
#ifdef __MINGW32__
buffer_append_strftime(out, "%Y-%b-%d %H:%M:%S", localtime64_r(&ent->mtime, &tm));
@@ -1002,6 +1011,7 @@ static void http_list_directory_ent(buffer * const out, const dirls_entry_t * co
buffer_append_string_encoded(out, name, ent->namelen, ENCODING_REL_URI_PART);
buffer_append_string_len(out, CONST_STR_LEN("/\">"));
buffer_append_string_encoded(out, name, ent->namelen, ENCODING_MINIMAL_XML);
+// buffer_append_string_len(out, CONST_STR_LEN("</a>/</td><td class=\"m\" data-value=\""));
buffer_append_string_len(out, CONST_STR_LEN("</a>/</td><td class=\"m\">"));
http_list_directory_mtime(out, ent);
@@ -1020,6 +1030,7 @@ static void http_list_file_ent(buffer * const out, const dirls_entry_t * const e
buffer_append_string_encoded(out, name, ent->namelen, ENCODING_REL_URI_PART);
buffer_append_string_len(out, CONST_STR_LEN("\">"));
buffer_append_string_encoded(out, name, ent->namelen, ENCODING_MINIMAL_XML);
+// buffer_append_string_len(out, CONST_STR_LEN("</a></td><td class=\"m\" data-value=\""));
buffer_append_string_len(out, CONST_STR_LEN("</a></td><td class=\"m\">"));
http_list_directory_mtime(out, ent);
Instead, the date strings are converted once to a data-value when the date column is clicked to be sorted, and then the data-value is stored in the element for use by future sorting actions.
Updated by gstrauss 3 months ago
- Status changed from Patch Pending to Fixed
Applied in changeset 4580edbf7fd8994aaccb7ecadf19a767409acc6f.
Also available in: Atom