Paul Stovell says…

As many people know, LINQ to Objects is made possible via extension methods. A two-second simple example of the where enumerable might look like this:

public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> predicate)
{
    foreach (T sourceElement in source)
    {
        if (predicate(sourceElement))
        {
            yield return sourceElement;
        }
    }
}

However, this leads to a point of confusion. Many people assume that to use extension methods or LINQ, all of the logic of the extension has to be implemented within one great big method.

This is simply not true. The SyncLINQ source, for example, isn’t one 40,000 line C# class with thousand-line methods. I’d bet that the LINQ source code isn’t either.

Instead, a common pattern for implementing LINQ extensions is to implement IEnumerable<T>, and to just return an object in the extension:

// The "Where" extension:
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> predicate)
{
    return new WhereIterator<T>(source, predicate);
}

// A class with all of the logic:
public class WhereIterator<T> : IEnumerable<T>
{
    private IEnumerable<T> _sourceCollection;
    private Func<T, bool> _predicate;

    public WhereIterator(IEnumerable<T> sourceCollection, Func<T, bool> predicate)
    {
        _sourceCollection = sourceCollection;
        _predicate = predicate;
    }   

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public IEnumerator<T> GetEnumerator()
    {
        foreach (T sourceElement in source)
        {
            if (predicate(sourceElement))
            {
                yield return sourceElement;
            }
        }
    }
}

While in this example is might not be necessary, it is when you are doing more complicated things than an a loop and a branch.

Pipes and Filters

Ayende has been blogging about a class design for implementing pipes and filters.

• Pipes and Filters: The IEnumerable approach
• Pipes and Filters: The multi threaded version
• Fluent Pipelines

In the comments, a few people suggested to use LINQ, or at least, extension methods. In Fluent Pipelines, Ayende writes:

Why do I use the first approach then?

Scalability.

What we are seeing here is about as trivial as it can get. What happens when we have more complex semantics?

Let us take writing to the database as an example. We need to do quite a bit there, more than we would put in the lambda expression, certainly. We can extract to a method, but then we run into another problem, we can’t do method inheritance. This means that I have no easy way of abstracting the common stuff outside. Well, I can use template method, but that works if and only if I have a single place I want to change behavior.

He’s totally right - the extensions are going to become complex, and quickly. However, I don’t think they have to. Using the sample I showed above, you could:

• Use inheritance
• Compose classes
• Use multiple threads within the class
• Not yield items, split items, transform them into different types

By adding extension wrappers, your code becomes much easier to understand. Once you’ve built the building blocks, you don’t have to write this:

public class TrivialProcessesPipeline : Pipeline<Process>
{
    public TrivialProcessesPipeline()
    {
        Register(new GetAllProcesses());
        Register(new LimitByWorkingSetSize());
        Register(new PrintProcessName());
    }
}

Instead, you can just write:

GetAllProcesses().LimitByWorkingSetSize().PrintProcessName()

I think this is much more "fluid". In fact, it looks exactly how a pipeline should look

Icebergs

Aside from sinking the Titanic, icebergs are famous for another thing: being much bigger underneath than what you can see on the surface.

Exposing an API as a set of LINQ extensions, using the approach I showed above, allows your library to work very much like an iceberg.

For example, the SyncLINQ library currently has:

• Two public classes
• Four public interfaces
• Forty nine internal classes
• Seven internal interfaces

Of the two public classes, the first is the Extensions class, which contains all of the LINQ extension methods (each method being just one line of code), and the second is an Exception class. All of the work is done behind the scenes, in internal classes, which are totally hidden.

Filed under: C#