Virtual List VIew (VLV) and Active Directory - What's it Good For?
One of the lesser-understood topics in working with Active Directory and LDAP is the Virtual List View, or VLV. In a nutshell, VLV lets you, the LDAP application developer, query a very large directory container in efficient, bite-sized chunks. Consider, for example, a directory with one very large "Users" OU; e.g.:
dc=demo,dc=local
+ ou=DemoGroups (50,000 groups)
+ ou=DemoUsers (50,000 users)
If you'd like to present the contents of this OU in a scrolling or paged window in an application, retrieving 50 pages of 1,000 results each is probably not the most efficient option; this will tax the directory server, waste network traffic, and suck up a lot of memory in your app too (especially if you aren't picky about retrieving just the necessary attributes).
VLV aims to solve this problem by providing a way to seek to positions within a very large LDAP result set, in conjunction with a sorting rule. Want to show just the first, or fiftieth, set of 20 users in an OU, ordered by displayName? Then it's time for VLV! We use Virtual List Views in Connect's web and mobile interfaces to ensure a reliable, responsive user experience when when browsing large directories... on the other side of the coin, Windows' Active Directory Users & Computers administrative tool should use VLV when possible, but doesn't--it'll just complain when an OU has more than 2,000 child objects and gives up.
This is actually a pretty common problem we see in LDAP client applications; often time the developer has built and tested against a small directory, and/or tests with highest privileges to avoid LDAP administrative limits.
Here's a screenshot of Connect using Virtual List Views over AD to provide a very efficient slider control. Even though the OU has several thousand objects, clicking or dragging the slider provides nearly instantaneous results (about 70 milliseconds per click):
;)
When including a VLV Request control in an LDAP query, we need to tell the directory server:
- How to sort the result set (without ordering, the idea of a "page" makes little sense)
- The position of the target entry (i.e., where in the list are we)
- How many entries before the target entry to fetch
- How many entries after the target to fetch
- An estimate of how many objects we think are in the view
Of particular note, the ordering rule is expressed by including a Sort Request Control. And #5, the estimated content count, helps the server seek to approximately the correct place in the result set. For example, if we ask the server for offset 500 in a view we think has 1,000 entries, but the directory data have changed so much that there are now 1,500 entries in the view, the server can adjust its answer to provide results at the position we probably wanted.
A couple other things to keep in mind when using VLV are that it works best when the sort control and LDAP filter line up; e.g., if we're sorting on the displayName attribute, then "(&(displayName=*))" is often a good search filter. Also, a "one level" LDAP search usually makes the most sense in conjunction with VLV, although a subtree search may sometimes do the right thing. If you issue a complicated or very specific filter along with a VLV control, the results may not quite be what you expect. Of course, you'll want to confirm that the attribute is VLV indexed by checking the attribute's searchFlags settings in the Schema container. Here's a quick way to find all the subtree (VLV) indexed attributes using Joe Richards' excellent adfind -- this'll find both one-level indexed attributes (searchFlags & 2) and subtree indexed (searchFlags & 64):
adfind -schema -flagdc -bit -f searchFlags:OR:=66 searchFlags ldapDisplayName
You might know the subtree index flag better from the Active Directory Schema Management console, where it's labeled as "Index this attribute for containerized searches" (is "containerized" a word? But I digress):
The Zetetic.Ldap project, available on Github and Nuget, provides code you can use directly or as a baseline for writing your own VLV requests.
Finally, here's a table of sample results on an Active Directory 2008R2 domain controller with some reasonably large OUs and various VLV options. This will help to give you a general idea of what VLV results to expect when applying specific vs. very general LDAP searches, using attributes that are or aren't indexed for VLV support. I'll just reiterate that this should support the conclusion that VLV works best on searches for an OU's immediate children, sorted and filtered on an attribute that has the subtree index bit set.