Project

General

Profile

Feature #1221

support *(wild card character) within include directive

Added by joungkyun about 10 years ago. Updated over 1 year ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
core
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Missing in 1.5.x:
No

Description

This patch support to use *(wild card character)within include directive.
For example:

include "conf.d/*.conf"

If you upper configration, and file not exists "conf.d/*.conf" or directory is not exists, lighttpd pass with no error messages.

Associated revisions

Revision bd66026f (diff)
Added by gstrauss over 1 year ago

[config] support include file glob (fixes #1221)

x-ref:
"support *(wild card character) within include directive"
https://redmine.lighttpd.net/issues/1221

History

#1 Updated by darix about 10 years ago


include_shell "cat conf.d/*conf" 

that is not enough?

#2 Updated by joungkyun about 10 years ago

Replying to darix:

{{{
include_shell "cat conf.d/*conf"
}}}

that is not enough?

Yes, not enough. I think that include_shell is not definite, and I need no error when conf.d/*.conf files not found.

#3 Updated by Anonymous over 9 years ago

This is really needed so that web application packages (rpm/dpkg/whatever) can put a file containing their configuration details in the conf.d directory and have it picked up by the web server automatically.

-- DavidDick

#4 Updated by stbuehler over 9 years ago

If the directory may be not containing files matching the mask, you just need to write a script instead of using cat.

#5 Updated by joungkyun over 9 years ago

Replying to stbuehler:

If the directory may be not containing files matching the mask, you just need to write a script instead of using cat.

First, sorry for my poor english.

I thinks that this is only developer's mind. As a system engienneer, include shell is very discomfort on trouble shooting. It's not clean. If I have situation of other system trouble shooting that is not mine, I will check user define scripts. And if this scripts has very dirty code, maybe I will forgive. :-)

As a sysstem engineer, I want to be removed include_shell option. :-) Almost of lighttpd users is not developer, and they have not skill of trouble shoot themselves.

So, I think that there is other way instead of include_shell, must use other way. include_shell is the only option for a geek.

On this case, I want to say deep discussion. But I stop this thread cause of my poor english. :-(

#6 Updated by stbuehler over 9 years ago

To search the directory conf.d for *.conf files and include them you could use this:


include_shell "find conf.d -maxdepth 1 -name '*.conf' -exec cat {} \;" 

And i disagree: if you cannot code such simple scripts, you should not be a server admin (at least not on a server which needs this feature).

I don't say that we don't want this feature - but applying such a patch needs time, and there are many more imporant things on my todo list.

Btw: The debian guys coded a perl script to include "enabled" modules as apache does.

And many people got problems as they used this system. It may be good for apache, but i wouldn't recommend it with lighttpd.

#7 Updated by joungkyun over 9 years ago

Replying to stbuehler:

And i disagree: if you cannot code such simple scripts, you should not be a server admin (at least not on a server which needs this feature).

So, I said that is only developer's mind. There is so many servers that have no server admin. I don't agree yours, too.

But, I agree that this patch is useless on lighttpd. And I don't want to fix it official tree with this patch. On case that need this patch, anybody use this patch.

This patch is for somebody who need this patch for their some case.

This is difference of a point of view between you and me. I think that you are not wrong and I'm not wrong.

#8 Updated by stbuehler almost 9 years ago

  • Status changed from New to Fixed
  • Resolution set to wontfix

As we agree we don't need it upstream, i close this.

#9 Updated by stbuehler almost 9 years ago

  • Status changed from Fixed to Wontfix

#10 Updated by alive almost 8 years ago

Pardon my french, but...
Flagging this as wontfix is stupid. This is a usability issue and should be adressed as such - there's no need to scare off sysadmins who want a clean script-less config file that includes some wildcarded files.

I think this should be prioritized. If you scare off people from implementing 'dumb shit' (as what this is seen as from some peoples perspective), you're gonna end up with a theoretically cool product that nobody uses because it's completely unusable at any scale other than benchmarking.

There's some people here want to be running lighty as part of big vhosted solutions but can't because the configuration file is extremely limited in a very flexible fashion...

Please reopen this and implement wildcard includes.

#11 Updated by rush over 7 years ago

I modified the patch to apply against 1.4.24. Please add my vote as I also think including upstream is a very good idea.

#12 Updated by soletan over 7 years ago

rush wrote:

I modified the patch to apply against 1.4.24. Please add my vote as I also think including upstream is a very good idea.

Even though I'm capable of coding stupid find invocations myself and even though I'm not considering everyone to be a geek in coding shell scripts I think adding glob support to include statement is improving usability of lighttpd.

I'm evaluating lighttpd for replacing existing Apache-based web server hosting several customers' sites with at least one configuration file per customer defining a virtual host each. Using the mentioned way of invoking find it's possible to include all pieces of configuration located in a subfolder. But what if any of the files (consider having 500+ files there) is including a typo and even stbuehler isn't free of typos 24/7. Then you get a perfectly nonsense error message like this:

2010-01-03 17:26:04: (configfile.c.867) source: find /etc/lighttpd/vhosts -name '*.conf' -exec cat {} \; line: 2 pos: 1 parser failed somehow near here: (EOL)

If lighttpd would parse folders itself there would be the name of a file. There might be opportunities to have an external parser, but that might be validating syntax, only. It won't validate semantic errors without cloning full lighttpd binary.

I'm sorry if integrating glob matching is such a hurdle for lighttpd development. But what is developing lighttpd good for? Is it just to consider oneself being a geek? I thought it's about efficiency and economically running a webserver ... then developers should care about CPU load as well as about simplifying usability.

EDIT: If using include_shell in the way mentioned before the usability is reduced even further by requiring proper end-of-line marker in last line of each and every file or otherwise resulting in an unsupported concatenation of configuration files lacking linebreak in between.

#13 Updated by icy about 7 years ago

  • Missing in 1.5.x set to No

Just for information: support for glob() style wildcards for "include" was added to lighttpd 2.0 some time ago

#14 Updated by hillct about 7 years ago

For what it's worth, it seems to me this functionality should be added to the 1.4 tree as well, if for no other reason than to better position lighttpd as a closer-to-drop-in replacement for Apache. While I can understand the argument that this feature is an administrative convenience and not critical functionality in that it can be scripted around, arguably, anything this simple, that can enhance the ease-of-deployment/migration for the software is a significant positive for the project as a whole.

When we migrated, several years back, I found the lack of this wildcard support to be an annoyance that we were able to work around, but there is always a percentage of administrators who elect not to migrate to lighttpd, based upon the overall number of changes required, to their system configuation so eliminating one more such required change has a marginal benefit that significantly outweighs the marginal cost of implementing within the 1.4.x tree.

Just my 2 cents.

#15 Updated by vpetersson almost 7 years ago

+1 for hillct.

The priority for this should really be bumped up. It makes the sysadmin's life a whole lot easier. Various find/cat solutions is just an ugly workaround (and it's asking for trouble when it comes to troubleshooting).
Also, it's already supported by both Apache and Nginx. Back-porting this to the stable branch should be a priority....at least in my world.

#16 Updated by nitrox over 6 years ago

You can´t backport from 2.0 really as it´s a complete rewrite.

#17 Updated by pwhack almost 2 years ago

vpetersson wrote:

+1 for hillct.

The priority for this should really be bumped up. It makes the sysadmin's life a whole lot easier. Various find/cat solutions is just an ugly workaround (and it's asking for trouble when it comes to troubleshooting).
Also, it's already supported by both Apache and Nginx. Back-porting this to the stable branch should be a priority....at least in my world.

As a RHEL & CentOS shop, I agree wildcard globbing should have been backported to 1.4.x branch. Yes, Red Hat has ancient versions of packages (e.g., the latest version of RHEL/CentOS (7) still has lighttpd 1.4.35). Aside from that any systems admin or engineer worth his or her salary is leveraging configuration management systems (Puppet, Salt, Chef, etc.) as much as possible. The fastest way to drop in a vhost file in Apache and Nginx is simply ensure the vhost file exists in the correct directory. Lighttpd does not support that in the 1.4 branch, which is the only "native" version available in RHEL (via EPEL no less). One could attempt to use config mgmt-supported providers/modules but all too often the author decides to implement the features he or she was looking for and not parameterize the whole package. So I often have to write my own modules. This is time consuming. As I said earlier, the fastest way is have a glob include and just have config mgmt ensure the various vhost files exist. Since lighttpd doesn't offer that on the platform my superiors have chosen, lighttpd isn't worth my time. Nginx is enjoying increased market share for many, good reasons. Now, off to replace lighttpd.

#18 Updated by gstrauss over 1 year ago

joungkyun took the time to attempt to address this and provided the patch for everyone's benefit, so I'd like to thank joungkyun for that effort. That said, the patch needs improvement. joungkyun's patch:
  • does not detect all glob metachars ("*?[]"), only "*"
    (and "{}" if GLOB_BRACE is available)
  • GLOB_BRACE is a GNU extension and might not be portable to other platforms
  • attempts to glob() twice, once with and once without context->basedir,
    which might lead to odd behavior or security issue depending on directory
    in which lighttpd is started and if full path to config file is provided
  • gratuitous print-line debugging code left in patch
  • gratuitous commented-out test code left in patch
  • gratuitous changes to code without documenting why
    (why is token parsing extended to allow '/' char in key?)
  • uses malloc and then sprintf (unsafe function) to concatenate two strings
    (instead of using safer buffer.h routines)

I have attached an updated patch against 1.4.39. Feedback appreciated.

#19 Updated by gstrauss over 1 year ago

  • Status changed from Patch Pending to Fixed
  • % Done changed from 0 to 100

Also available in: Atom