Netresolve is a package for nonblocking network name resolution via backends intended as a replacement for name service switch based name resolution in glibc.
Pavel Šimerda pavlix at pavlix.net (mail and jabber) pavlix at IRC Freenode
The classic way:
./autogen.sh
make
make install
Build RPM package:
./autogen.sh
make rpm
- File descriptor based nonblocking host/service name resolution API
- suitable for various file descriptor based event loops
- unlike glibc or POSIX APIs
- Extensible request and result APIs
- unlike glibc or POSIX APIs
- host/service queries (like
getaddrinfo) - SRV record support for host/service queries
- reverse queries (like
getnameinfo) - DNS style queries (like
res_querybut via backends)
- Backend-based network name resolution somewhat similar to glibc's nsswitch
- more flexible than DNS-only libraries
- with configurable backend options (not available in nsswitch)
- also supports nsswitch backends (via
libnetresolve_nss.so)
- Avoids limitations and bugs found in glibc
- support for
ifindexandscope_id(problematic in nsswitch) - support for TTL information (problematic in nsswitch)
- support for
- DNS happy eyeballs implementation
- concurent A/AAAA requests
- quick timeout when there's no answer to one of A/AAAA requests
- Socket API
- callback based wrappers over
socket(),bind()andconnect() - the application receives a successfully bound or connected socket
- callback based wrappers over
- TCP happy eyeballs implementation
- concurrent IPv4/IPv6 connect
- quick timeout when there's no answer to one of IPv4/IPv6 TCP SYN packets
- Security information
- well-known and locally configured data is considered secure
- experimental support for DNSSEC authenticated data
The command line tool is useful for testing the netresolve library as well as testing various libraries supported by netresolve including glibc and libunbound.
Forward query:
netresolve --node localhost
netresolve --node www.sourceware.org --service http --protocol tcp
Forward query with SRV resolution:
netresolve --srv --node jabber.org --service xmpp-client --protocol tcp
Reverse query:
netresolve --address 192.228.79.201
netresolve --address 2001:503:ba3e::2:30
netresolve --port 80
netresolve --address 192.228.79.201 --port 80
DNS style query:
netresolve --type srv --node _xmpp-client._tcp._jabber.com
netresolve --class ch --type txt --node version.bind
Connection support similar to netcat or socat:
netresolve --connect --node localhost --service 22
Query information using a DNS backend and trust the AD flag:
netresolve --backends aresdns:trust --node www.dnssec.cz
Query information using a validating DNS backend:
netresolve --backends ubdns:validate --node www.dnssec.cz
Program or library using netresolve has to allocate a netresolve context (or context) which is then used to perform all the queries. You may want to create multiple contexts to have different configurations prepared for use. By default the netresolve context is in blocking mode. Consult the next section about event loop integration on how to create a netresolve context in nonblocking mode.
#include <netresolve.h>
netresolve_t context = netresolve_context_new();
Then you tweak the context configuration to your liking.
netresolve_set_protocol(context, IPPROTO_TCP);
netresolve_set_dns_srv_lookup(context, true);
When you're happy with the configuration, you can run your queries using netresolve_query_forward(), netresolve_query_reverse() or netresolve_query_dns(). Any of the functions waits until the query is successfully finished, failed or timed out.
netresolve_query_t query = netresolve_query_forward(context, "www.sourceware.org", "http", NULL, NULL);
TODO: Error reporting.
You can pick up your query immediately.
size_t count = netresolve_query_get_count(query);
for (size_t idx = 0; idx < count; idx++) {
int family;
const void *address;
int ifindex;
int socktype;
int protocol;
int port;
int priority;
int weight;
uint32_t ttl;
netresolve_query_get_node_info(query, idx, &family, &address, &ifindex);
netresolve_query_get_service_info(query, idx, &family, &address, &ifindex);
netresolve_query_get_aux_info(query, idx, &family, &address, &ifindex);
/* do something with the data */
}
You can free the query object when you no longer need it.
netresolve_query_free(query);
When you're not going to use the context to perform any queries nor use the existing query objects to examine the results, you can free the context together with all queries that haven't been freed as written above.
netresolve_context_free(context);
An example of using netresolve in blocking mode can be found in test/test-sync.c.
The nonblocking mode is designed to be independent of a specific event loop implementation. You can use one of the existing event loop connectors or write your own easily. Connectors for libevent and glib are distributed as header files to avoid additional dependencies. Connectors using epoll-style and select-style file descriptor sets are built into the library.
Provided that you have a struct event_base *base pointer to the event base, create the netresolve context before issuing any queries.
#include <netresolve-event.h>
netresolve_t context = netresolve_event_new(base);
Free it as usual.
netresolve_context_free(context);
With glib you can easily create a channel attached to the default context.
#include <netresolve-glib.h>
netresolve_t context = netresolve_glib_new();
Free it as usual.
netresolve_context_free(context);
Issue a query with a callback and user data.
netresolve_query_t query = netresolve_query_forward(context, "www.sourceware.org", "http", callbac, user_data);
Pickup the data in your callback and free the query if you no longer need it.
void
callback(netresolve_query_t query, void *user_data)
{
size_t count = netresolve_query_get_count(query);
for (size_t idx = 0; idx < count; idx++) {
int family;
const void *address;
int ifindex;
int socktype;
int protocol;
int port;
int priority;
int weight;
uint32_t ttl;
netresolve_query_get_node_info(query, idx, &family, &address, &ifindex);
netresolve_query_get_service_info(query, idx, &family, &address, &ifindex);
netresolve_query_get_aux_info(query, idx, &family, &address, &ifindex);
/* do something with the data */
}
netresolve_query_free(query);
}
Note: You can use callbacks with blocking mode as well, although it's not as useful as with nonblocking mode. This feature is especially useful in code that is written to work with both blocking and nonblocking mode.
Create the context.
#include <netresolve-epoll.h>
netresolve_t context = netresolve_epoll_new();
Retrieve the file descriptor.
int fd = netresolve_epoll_fd(context);
When the epoll file descriptor is ready ready for reading, dispatch.
netresolve_epoll_dispatch(context);
Free it as usual.
netresolve_context_free(context);
Create the context.
#include <netresolve-select.h>
netresolve_t context = netresolve_select_new()
Retrieve the file descriptors.
fd_set rfds, wfds;
int nfds;
FD_ZERO(&rfds);
FD_ZERO(&wfds);
nfds = netresolve_select_apply_fds(context, &rfds, &wfds);
When a file descriptor in rfds is ready for reading, dispatch.
netresolve_select_dispatch_read(context, rfds);
When a file descriptor in wfds is ready for writing, dispatch.
netresolve_select_dispatch_write(context, rfds);
Free it as usual.
netresolve_context_free(context);
If none of the included integration functions match your needs, you can create the context using netresolve_context_new() and then attach your own set of callbacks using netresolve_set_fd_callbacks(). You need to provide two callbacks. The watch_fd() function adds an event source consisting of a file descriptor, and a set of events (subset of POLLIN | POLLOUT) and an opaque pointer data and the unwatch_fd() function that removes the source. When a file descriptor event occurs, the event loop implementation calls netresolve_dispatch() with the respective source and the subset of events that occured. The integration code can optionally provide a user_data pointer that can ten be retrieved with netresolve_get_user_data() and a free_user_data() callback that will be called during the context destruction and a pointer for each event source returned by watch_fd() that will then be passed to unwatch_fd() as the handle argument. The user_data pointer typically points to an object representing the event loop and the handle pointer points to an object representing the event source.
void *watch_fd(netresolve_t context, int fd, int events, netresolve_source_t source);
void unwatch_fd(netresolve_t context, int fd, void *handle);
void free_user_data(void *user_data);
Install callbacks.
netresolve_set_fd_callbacks(context, watch_fd, unwatch_fd, user_data, free_user_data);
Dispatch an event.
netresolve_dispatch(context, source, events);
Use one context object per thread. Avoid accessing the context and query objects from different threads for now.
You can use a compatibility API most resembling the POSIX one but still allowing for nonblocking mode. The context object must be created as usual and you can also tweak its configuration and set up nonblocking mode and callbacks. This API can be nonblocking depending on the context configuration already described.
#include <netresolve-compat.h>
A special query constructor lets you use input arguments as with getaddrinfo().
struct addrinfo hints = { .ai_family = AF_UNSPEC, .ai_socktype = 0, .ai_protocol = IPPROTO_TCP, .ai_flags = 0 };
netresolve_query_t query = netresolve_query_getaddrinfo(context, "www.sourceware.org", "http", &hints, callback, user_data);
A special query result getter lets you use output arguments as with getaddrinfo() and destroys the query object for your convenience.
struct addrinfo *result;
uint32_t ttl;
int status = netresolve_query_addrinfo_done(query, &list, &ttl);
When you don't need the result object any more, free it.
netresolve_freeaddrinfo(result);
An alternative getaddrinfo() implementation using netresolve in compat/libc.c serves as an example of how to use this compatibility API.
A backend for glibc nsswitch is provided as libnss_netresolve.so that exposes part of netresolve funcionality via the glibc name resolution API. The backend also supports a variant of _nss_*_getaddrinfo() API proposed by Alexandre Oliva.
If your program doesn't need the full power of the provided API but you still want to use netresolve as your resolver implementation, you can simply link your program to libnetresolve-libc.so which overrides selected libc name resolution functions.
Supported functions:
getaddrinfo()getnameinfo()gethostbyname(),gethostbyname_r()gethostbyname2(),gethostbyname2_r()gethostbyaddr()
You can use the wrapresolve command to start a program with LD_PRELOAD configured so that your program uses netresolve functions instead of libc.
wrapresolve curl http://www.sourceware.org/
You can tweak netresolve configuration via environment variables, or you can use convenience options provided by wrapresolve. Currently there are options to force IPv4-only or IPv6-only resolution.
wrapresolve -4 curl http://www.nix.cz/
wrapresolve -6 curl http://www.nix.cz/
The netresolve package also provides a couple of testing tools for the above libc functions that can be used to test both libc and the replacement functions.
getaddrinfo --node localhost --service http
wrapresolve getaddrinfo --node localhost --service http
The wrapresolve command also pulls in libnetresolve-asyncns.so which implements libasyncns API using netresolve.
Two non-blocking APIs wrapping name resolution together with the socket(), bind(), listen() and connect() calls are available.
The simpler one is netresolve_listen() that creates listening sockets for all available addresses and passes back sockets for accepted connections. You can stop the listening sockets and free the structures using netresolve_listen_free().
The more sophisticated one is netresolve_connect() that uses the list of addresses to get a single connected socket, maintaining address preference by family and other characteristics, and using a short timeout to bypass the preference when it would take too much time. A function called netresolve_connect_next() can be used to overcome application level issues with one of the addresses and to get a new connection using the next available address. Once happy with the connected socket or to abort the process, run netresolve_connect_free().
The list of backends can be chosen using netresolve_set_backend_string() or via the NETRESOLVE_BACKENDS environment variable. Backends are separated by a comma and accept options separated by a colon. A plus sign prepended to the backend name can be used to run that backend even if another backend already succeeded.
export NETRESOLVE_BACKENDS=any,loopback,numerichost,hosts,hostname,avahi,aresdns
The command line interface also supports an option to select the backend.
netresolve --backend any,loopback,numerichost,hosts,hostname,avahi,aresdns
The default list of backends is slightly wider then the example one above and attempts all sorts of name resolution tools in order to give you full results.
Three backends, any, loopback and numerichost, are available that perform trivial translations. The hosts backends uses /etc/hosts database of nodes. Nonblocking API is most useful for remote services. We have two nonblocking DNS backends, aresdns and ubdns. We support special configuration of the two DNS backends, aresdns:trust reads the DNS AD flag and marks the query result secure and ubdns:validate instructs libunbound to perform the validation. On systems with Avahi service running, the avahi backend offers Multicast DNS name resolution.
You can ask netresolve to call getaddrinfo() to gather the data using the getaddrinfo backend. This is useful for testing the libc API as well as comparing results of general purpose netresolve backends to other implementations. This backend blocks until the getaddrinfo() function exits.
netresolve --backends getaddrinfo --node www.sourceware.org
We also support glibc nsswitch modules through the nss backend. It finds the nsswitch dynamic module by name, loads the supported API functions and runs the most suitable one. The algorithm used in netresolve was inspired by glibc getaddrinfo() and glibc nscd code. This backend blocks until the nsswitch module function exits.
netresolve --backends nss:files --node localhost
netresolve --backends nss:dns --node www.sourceware.org
You can also pass the nsswitch module by absolute or relative path.
netresolve --backends nss:/usr/lib64/libnss_files.so
Or you can select the specific nsswitch API to be used.
# call `_nss_files_gethostbyname_r`
netresolve --backends nss:files:gethostbyname --node localhost
# call `_nss_files_gethostbyname2_r`
netresolve --backends nss:files:gethostbyname2 --node localhost
# call `_nss_files_gethostbyname3_r`
netresolve --backends nss:files:gethostbyname3 --node localhost
# call `_nss_files_gethostbyname4_r`
netresolve --backends nss:files:gethostbyname4 --node localhost
You can use the nss plugin to test systemd-resolved via libnss_resolve.so.
netresolve --backends nss:resolve --node localhost
We have experimental support for new nsswitch API based on a variant of getaddrinfo() proposed by Alexandre Oliva. This proposed API would consolidate the nsswitch APIs and overcome some of the limitations describe at the beginning of this document.
There's an experimental backend to call a script to perform name resolution and feed it with the request serialized in a couple of text lines and read the result from it.
netresolve --backends exec:/path/to/my/script --node localhost
That one can be used for interactive testing as well:
netresolve --backends exec:socat:-:/dev/tty --node www.example.com
Note: This backend is untested and maybe not even functional.
When a name resolution request is ready, netresolve calls one of the setup functions depending on the query type. If any of the setup functions is missing, netresolve assumes that the backend doesn't support the respective query type at all.
setup_forward(context, settings);
setup_reverse(context, settings);
setup_dns(context, settings);
The setup function initializes the query using input data from netresolve_backend_get_*() functions. It creates a state object using netresolve_backend_new_priv() (if necessary) to share its data with the other backend API functions. Finally it does one of the following:
- Add any data items using
netresolve_backend_add_*()functions and callnetresolve_backend_finished()to signal immediate success. - Call
netresolve_backend_failed()to signal immediate failure. - Add one or more watchers using
netresolve_backend_watch_fd()andnetresolve_backend_watch_timeout().
In the third case, a dispath function is called by netresolve. It is not needed in the other two cases.
dispatch(context, settings);
The dispatch function retrieves the state object using netresolve_backend_get_priv(). Then it does one of the following:
- Add any data items and call
netresolve_backend_finished()to signal success. - Call
netresolve_backend_failed()to signal failure. - Leaves one or more watchers active.
When success or failure was reported by the plugin, or when the query has been cancelled, the cleanup function is called.
cleanup(context, settings);
The cleanup function also retrieves the state object, removes remaining watchers using netresolve_backend_watch_fd() and/or netresolve_backend_remove_timeout(), closes open file descriptors, returns allocated memory (except the context object itself which is freed by netresolve itself).
Calls to the above functions in a single backend are serialized, calling a backend API function doesn't cause any side effects for the backend.
The library is still considered experimental. The functions in netresolve.h are getting stable very soon.
The library doesn't currently sort paths according to their priority and/or RFC rules. The c-ares library blocks when /etc/resolv.conf is empty instead of quitting immediately, which in turn breaks tests when offline. The DNS backend doesn't support search domains. For more information, see the TODO file.
- Tore Anderson was the first who discussed name resolution issues with me and especially the reasons to suppress A or AAAA queries when global address of the respective family is not present in the system.
- Simo Sorce told me about
getaddrinfo()issues with canonical names. - Ulrich Drepper wrote a good amount of documentation on network name resolution.
- Carlos O'Donnel helped me to find my way in the glibc community.
- Nick Jones brought in the idea of a file descriptor based nonblocking API.
- Petr Špaček pointed me to SSSD developers.
- Jakub Hrozek told me about c-ares DNS library and about SSSD name resolution requirements.
- Miloslav Trmač reminded me of the need to also provide binding and connecting facilities on top of name resolution.
- Lennart Poettering wrote libasyncns, an async wrapper over libc resolver functions
- Alexandre Oliva wrote a number of proposals to improve glibc name resolution
One way to look at netresolve is to see it as a testbed for future glibc improvements. It's written with testing and debugging in mind and prototyping new ideas in netresolve is very easy. Another way is to see it as a more flexible alternative to glibc host and service name resolution functionality. Either way glibc and POSIX were the main inspirations for this project.
They provide ares_fds() and ares_timeout() to update the current set of file descriptors and the associated timeout. The netresolve callback API improves greatly on that. The application submits the watch_fd() callback.
They provide a backend-based caching service for user/group name resolution, authentication and related stuff. As netresolve doesn't really need to cache its results nor does it need to perform authentication services, it provides just a shared library, not a long-running daemon. Also, for DNS resolution they currently use c-ares.
According to the website, libevent is using internal asynchronous DNS as well. The same applies to squid. Apparently the operating system's resolution API is not sufficient for many tools.
To my knowledge, name resolution related systemd features were announced after I published netresolve. The relation to that project is yet to be seen.
Copyright (c) 2013 Pavel Šimerda, Red Hat, Inc. (psimerda at redhat.com) and others All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.