We recently released the latest preview release of MVC 3, including an updated version of Razor.  One of the things we did in this release is simplify the hosting APIs dramatically.  I did a demo of these new APIs in a pre-recorded PDC10 talk I did with Scott Hunter, which is available to stream here: http://bit.ly/ac7B0P.  As I promised in that talk (and a few others before and after it ), I'm finally blogging about the sample I showed in that talk!

The hosting APIs center around two classes:

• RazorTemplateEngine: the primary entry point to the Razor engine
• RazorEngineHost: defines the environment in which the generated Razor code will run

First, let's look at the Host.  RazorEngineHost is a class with a number of useful properties on it that can tweak the way the generated code behaves.  More interestingly (though also more complicated and a topic for later), it can be subclassed and used to swap out the parser and code generator with your own custom code to add new keywords and features to Razor for your scenario!  In the sample I showed at PDC, we just used the properties to configure things, like so:

// Set up the hosting environment
// a. Use the C# language (you could detect this based on the file extension if you want to)
RazorEngineHost host = new RazorEngineHost(new CSharpRazorCodeLanguage());
// b. Set the base class
host.DefaultBaseClass = typeof(TemplateBase).FullName;
// c. Set the output namespace and type name
host.DefaultNamespace = "RazorOutput";
host.DefaultClassName = "Template";
// d. Add default imports
host.NamespaceImports.Add("System");
host.NamespaceImports.Add("System.IO");

This code is just doing fairly simple (but powerful) stuff.  It first sets the base class for the template.  Since we are running Razor outside of ASP.Net, you don't have to use our base classes.  In fact, you have to provide your own!  The sample contains a simple base class which just writes the template content to a StringBuilder we can retrieve later.

Razor generated code using a certain convention (that can be overridden if necessary).  The generated template will contain a public method called "Execute" and that method will contain the users code and calls to one of the following methods:

• Write() - When a user types "@foo", we generate Write(foo) in the generated code
• WriteLiteral() - When a user types literal markup, like "<p>Foo</p>", we generate WriteLiteral("<p>Foo</p>") in the generated code

The names of these methods can be configured using the GeneratedClassContext property of the host, but we won't go into details on that here.  If you want to enable features like Sections, Templates and Helpers in your custom Razor host, you need to set some properties here but we'll cover that another time.

The host also defines the namespace and class name of the generated C# or VB code, and the list of namespaces to be imported into the template's generated code file.  Once we've configured our host, we construct a RazorTemplateEngine and give it this host to use:

// Create the template engine using this host
return new RazorTemplateEngine(host);

From there, we're set to generate some code!  Let's look at what happens when you click the "Load Template" button in the sample.

// Generate code for the template
GeneratorResults razorResult = null;
using (TextReader rdr = new StringReader(templateTextBox.Text))
{
    razorResult = _engine.GenerateCode(rdr);
}

Now that we have our engine configured, all we need to do is take the template source code (from a text box in this case), open up a TextReader for it, and pass it in to the RazorTemplateEngine we constructed earlier.  The result of that call is a structure called GeneratorResults which contains the following properties:

• Success - A boolean indicating if parsing and code generation were successful
• GeneratedCode - A CodeDOM tree representing the code that the Razor engine generated
• ParserErrors - A list of parser errors, if any
• Document - The root node of the Razor parse tree (we'll cover that later)

From this one call, you get all that data!  You can party on the parse tree, or use the CodeDOM APIs in the core .Net Framework to generate source code.  In the sample, I take the CodeDOM tree, compile an assembly using it, then load the assembly and the generated template type.  It takes quite a bit of code, so I'll leave that to the sample rather than posting it all here, but it's easily wrapped up in a helper method if you are going to be doing a lot of compiling!

It is important to note that Razor is not an interpreted language.  It actually follows the same compilation model as ASPX, despite the drastically different syntax.  The first time a user requests a Razor page, we generate the C#/VB code for it and compile it into an assembly.  All future requests, until the original file is changed, go against that compiled C#/VB code.  This means we can't easily do things like implementing "eval" because we aren't actually running the Razor code directly, we're just using it to generate C#/VB code.

Hopefully that gives you a quick summary of how to get your feet wet hosting Razor outside of ASP.Net.

Also, in MVC RC, we released Visual Studio support for Razor.  It's still an early release, so there might be some issues.  We'd like to hear from you if you encounter any issues!  Email razorvs@microsoft.com with your issues (related to VS tooling only please).  If you have other questions about Razor in general, tweet me at @anurse or email me at andrew AT andrewnurse DOT net!

Here's the sample: HtmlGen.zip (65.04 KB) (NOTE: MVC 3 RC is required).

I wanted to jump in with my $0.02 (Canadian ) on Microsoft.Data.  David Fowler, fellow ASP.Net team member, posted about it earlier today, and the response has been ... active .  The message behind most of these responses has been that it encourages bad practices to novice developers.  I think there's an important point that's being missed here: It doesn't matter how hard we work, as professional developers, to create clean architectures and abstractions, there's a whole world of novice developers who just don't care about that.  They want to write code now and be done with it.  This is the audience targeted by Microsoft.Data, and the WebMatrix product as a whole.  If novice developers come to the Microsoft platform and see these complicated architectures (which, don't get me wrong, have massive benefits for professional development), they simply won't adopt them, they'll just head to a different platform which allows them to get their job done quickly.

Microsoft.Data is, in my opinion, actually a step in the right direction.  For example, let's say I'm a novice developer, and I go out and look for samples and documentation in order to piece together a simple product list from my database.  As long as we do our work right (and we plan to), the documentation will lead me to write something like this:

@{ var products = db.Query("SELECT * FROM Products WHERE CategoryId = @0", categoryId); }
@foreach(var product in products) {
    <li>@product.Name</li>
}

It works, and it's SQL Injection safe.  Note that although David didn't blog directly about this, Microsoft.Data fully supports parameterized SQL and it actually supports it better than traditional ADO.Net (note that I don't have to fiddle with SqlParameter objects).

Now, I'm learning more about proper architecture, and I decide to switch to an ORM and use Linq for my queries.  A quick one-liner change and I'm all set:

@{ var products = db.Products.Where(p => p.CategoryId == categoryId); }
@foreach(var product in products) {
    <li>@product.Name</li>
}

[NOTE: WebMatrix doesn't include a true ORM, as there are plenty of good options out there, this is just a sample of how one could ramp up from Microsoft.Data to more powerful ORMs]

And if I want to go full-tilt and use ASP.Net MVC, I can move my data access code into a Controller (or even deeper into my architecture) and then a bit of copy-pasting gets me to:

public ActionResult Products(int categoryId) {
    var products = db.Products.Where(p => p.CategoryId == categoryId);
    return View(products);
}

@foreach(var product in Model) {
    <li>@product.Name</li>
}

From there I'm free to keep refactoring things behind further abstractions (Repository patterns, etc.). I've started simple and refactored as necessary to improve the architecture. In every step, I've been able to take a lot of the existing code with me, which traditional ADO.Net and other platforms (like PHP) don't make quite as easy.

The simple fact is this: The audience we're targeting is already using inline SQL, they are perfectly happy to keep doing so and they are not interested in clean abstractions (to the point of finding them complex and unnecessary).  Microsoft.Data, and the entire ASP.Net Web Pages framework (the inline page model used in WebMatrix), is an attempt to provide a simple model for web development that provides on-ramps to guide users towards best practices.

One of the features of Razor which hasn't been discussed a lot is the Inline Template feature.  Razor includes the ability to provide an inline Razor template as an argument to a method.  At the moment, this is only used by the Grid helper in ASP.Net Web Pages (and Scott Guthrie showed it back in his original blog post).  However, we don't have much documentation on how to create your own templated helpers yet, so I figured I'd talk a bit about it.

First, let's take a look at what code is generated when we use an inline template.  I've written a sample templated helper called "Repeat" which just repeats the content of the template a specified number of times (we'll take a look at the implementation later).  The page that uses this helper looks something like this:

<!DOCTYPE html>
<html>
     <head>
         <title>Repeat Helper Demo</title>
     </head>
     <body>
         <p>Repeat Helper</p>
         <ul>
             @Repeat(10, @<li>List Item</li>);
         </ul>
     </body>
</html>

And when we run it, we get the following output:

Let's take a look at the implementation of "Repeat".  I wrote it inline in the page in this case, but you could just as easily write it in a static class in App_Code and reference it that way.  In Razor, the "@functions" block lets you write code that will be injected as-is into the body of the generated class.

@using System.Text;
@functions {
      public static IHtmlString Repeat(int times, Func<int, object> template) {
           StringBuilder builder = new StringBuilder();
          for(int i = 0; i < times; i++) {
              builder.Append(template(i));
          }
          return new HtmlString(builder.ToString());
      }
}

So, the template is being passed in as a Func<int, object> and when we invoke it, we get back the result of running the template.  But, if you look at line 6, you'll notice we're passing in an argument to the template function.  Let's take a look at the C# that is generated when we write a call to Repeat.  Here's the generated code to match the Razor code in the first code listing:

this.Write(Repeat(10,item => new Microsoft.WebPages.Helpers.HelperResult(__writer => {
     @__writer.Write(" ");
     @__writer.Write("<li>List Item</li>");
})));

It's a little complex looking, but essentially, what's happening is that we're writing out a lambda which accepts a single parameter called "item" (the type of which is determined by the method you're passing it to).  When that lambda runs, we construct and return a HelperResult.  HelperResult is a class defined in the ASP.Net Web Pages framework, and it's essentially a wrapper around yet another delegate which writes text to a TextWriter.  Think of it as a mini Razor template, when you invoke the delegate, it writes the content of the template.  The advantage of wrapping the delegate up in the HelperResult class is that we can treat it just like a string in most places since it overrides ToString to return the result of executing the template.

The "item" parameter is used in helpers like the Grid helper to provide the current data item to the template so that it can be used.  The Repeat helper passes in the iteration number as this parameter, which we can access from within the template by using "@item"?.  For example:

@Repeat(10, @<li>List Item #@item</li>);

Which will render:

In summary, if you want to use Razor templates in your helper methods, just add a parameter with the type Func<?, object> where ? can be any type you want.  As an interesting little exercise, try converting the Repeat helper to take an IEnumerable<T> and pass each item of that enumerable to the template, rendering the result.

I've uploaded the Razor file containing my sample helper here: RepeatHelper.cshtml.txt (.56 KB) (Note: To avoid issues with file types on my hoster, it has a .txt extension, just remove that and you're good to go!)

Please feel free to ask questions in the comments, or by emailing me at andrew AT andrewnurse DOT net.  I'm also on twitter at @anurse and I keep an eye on the "razor" tag on StackOverflow so there's no shortage of ways to ask!

When Scott Guthrie originally blogged about Razor, he mentioned that it was fully hostable outside of ASP.Net.  The engine itself is not quite as detached from System.Web as we'd like, but it's close and we're going to get it way closer in the next release.

Having said that, you can still host Razor outside of the ASP.Net pipeline with the current beta! It's a little trickier, and you do technically need to reference System.Web.  I've written a sample console app that I'm attaching to this post called "rzrc" which takes in  a .cshtml or .vbhtml Razor file and runs it through the parser and code generator to produce a .cs or .vb file.  I'll walk through the main logic here and go over what each section does.

However, I was not the first to do this! Full credit for that goes to Gustavo Machado, who wrote an excellent post in which he used Reflector to work out how to run the Razor parser and code generator.  Well done Gustavo!  There are a few things that this version does that Gustavo's doesn't, such as cleaning up the Web-related stuff in the generated code and selecting the language based on the Code Language, but he basically hit it spot on!

The first thing my console app does is get the input file name, extract the extension and look up what Razor Code Language it uses.  This is done using the CodeLanguageService class, which is part of the Razor APIs:

CodeLanguageService languageService = CodeLanguageService.GetServiceByExtension(extension);
if (languageService == null) {
    Console.WriteLine("{0} is not a Razor code language", extension);
    return;
}

Then, we fire up the parser and the code generator.  A CodeLanguageService is basically a factory for constructing a Code Parser, to parse the code blocks after an "@" and a matching Code Generator to write the final C# or VB class.

InlinePageParser parser = new InlinePageParser(languageService.CreateCodeParser(), new HtmlMarkupParser());
CodeGenerator codeGenerator = 
    languageService.CreateCodeGeneratorParserListener(className,
                                                        rootNamespaceName: "Template", 
                                                        applicationTypeName: "object", 
                                                        inputFileName, 
                                                        baseClass: "System.Object");

When you run the Razor Parser, you must provide it with an object implementing IParserConsumer.  This interface has callbacks which the parser will call when it encounters various Razor constructs (more details on the Razor parse tree later).  CodeGenerator implements this interface and responds to the these callbacks by generating code.  However, it does nothing with the errors, so in the console app, I've written a very simple IParserConsumer called CustomParserConsumer which wraps the code generator and outputs errors to the console.  I won't put the code here, but it's in the sample, so take a look there if you're interested.

Now that we've got all the objects we need, we can actually run the parser over the input

CustomParserConsumer consumer = new CustomParserConsumer() { CodeGenerator = codeGenerator };
using (StreamReader reader = new StreamReader(inputFileName)) {
    parser.Parse(reader, consumer);
}

Once Parse returns, the Code Generator will have built a CodeDOM tree representing the generated code during the callbacks, so we know that our code is ready to go.  Right now, the Code Generator adds in some web specific things.  For example, when we constructed the Code Gneerator above, we gave it an "applicationTypeName" which (in a web context) is the type name of the class defined in Global.asax, if there is one.  Since we are trying to generate a template that isn't related to the web, we can get the CodeDOM tree from the Code Generator and remove these things.

codeGenerator.GeneratedCode.Namespaces[0].Types[0].Members.RemoveAt(0);
codeGenerator.GeneratedCode.Namespaces[0].Types[0].BaseTypes.Clear();
codeGenerator.GeneratedCode.Namespaces[0].Imports.Clear();

Finally, we use the CodeDOM to write the code to a C# or VB class file (provider is a CodeDomProvider from System.CodeDom.Compiler):

using (StreamWriter writer = new StreamWriter(outputFile)) {
    provider.GenerateCodeFromCompileUnit(codeGenerator.GeneratedCode, writer, new CodeDom.CodeGeneratorOptions());
}

And we're done!  This is definitely more complicated than we'd like, but there are plans to simplify this API significantly in future releases.  For the most part, all we've done is left the methods our ASP.Net Build Provider uses open and accessible.  I wouldn't bet on these APIs staying around too long, but any API changes from here on should be simplifications.  For now though, check out the sample I've attached and play around!  Note that you must have WebMatrix installed to use the sample. 

I've put some comments in which start with "EXT" which contain tips on how to extend this code to your own use.  Please feel free to take this code and use it absolutely anywhere you want!  Let me know how your using Razor by either tweeting me at @anurse or email me at andrew AT andrewnurse DOT net.

Download the console app here: rzrc.zip (3.46 KB)

This is part 2 of my Inside Razor series. Read Part 1 here.

In my previous post, I glossed over one line in my sample:

<li>@p.Name ($@p.Price)</li>

Well, it's finally time to get in to how this is parsed! So, to recap from last time, when we see the "<li>" here, we know that we are parsing a block of markup which ends at the "</li>". The markup parser scans forward until it finds the end tag, but before it reaches it, it sees an "@". So, just as with "@foreach", it switches to the code parser.

This is where things get a bit different. The C# code parser looks at that first identifier: "p" and checks its internal list of C# keywords. Of course, "p" is not a C# keyword, so the C# code parser enters "Implicit Expression" mode. The algorithm for parsing implicit expressions is something like the following:

1. First, Read an identifier
2. Is the next character a "(" or "["?
   • Yes - Read to the matching ")" or "]", then Go To 2
   • No - Continue to 3
3. Is the next character a "."?
   • Yes - Continue to 4
   • No - End of Expression
4. Is the character AFTER the "." a valid start character for a C# identifier?
   • Yes - Read the "." and Go To 1
   • No - DO NOT Read the ".", and End the Expression

The high-level overview of this algorithm is that an implicit expression is an identifier, followed by any number of method calls ("()"), indexing expressions ("[]") and member access expressions ("."). And, whitespace is not allowed (except for within "()" or "[]"). So for example, these are all valid implicit expressions in Razor:

@p.Name
@p.Name.ToString()
@p.Name.ToString()[6 - 2]
@p.Name.Replace("ASPX", "Razor")[i++]

However, the following are not valid, and the second section (after the arrow, "==>") is the only part that would be considered part of the expression by Razor:

@1 + 1 ==> @
@p++ ==> @p
@p    .   Name ==> @p
@p.Name.Length - 1 ==> @p.Name.Length

This is why we have another syntax for expressions: "@(...)". This syntax allows anything you want within the "()". So, you can write all of the previous examples using that syntax as an escape-hatch:

@(1 + 1) 
@(p++) 
@(p    .   Name) 
@(p.Name.Length - 1)

Once we've identified the expression, we pass it along to our code generator. When generating the code for "@foreach () { ... }", we just dump that code into the generated C# class as-is, but when we identify an expression (either implicit or explicit) we do something a little different. You probably noticed that unlike ASPX, there is only one control construct: "@", there is no "@=" to distinguish code that we run vs. expressions that we render the value of. This is where some of the magic of Razor comes in. If we see "@foreach" for example, we know that "foreach" is a C# keyword, so that block is written as a statement to be executed. When we see "@p.Name" or "@(1 + 1)", we know that they are expressions, so after executing them, we render the result. So basically:

• @if, @switch, @try, @foreach, @for, etc. are equivalent to "<% %>"
• @p.Name, @(p++), @(1 + 1), etc. are equivalent to "<%: %>"

Another side note is that expressions are equivalent to "<%:" and NOT "<%=". We made a decision in Razor that HTML encoding should be the default, and that if you want to write unencoded strings, you can use the IHtmlString interface that has been blogged about before.

So, with all that background, we can quickly jump back to our initial sample:

<li>@p.Name ($@p.Price)</li>

When we see "@p.Name" we identify that as an expression, but the space before the "(" stops us from interpreting it as a method call. Then " ($" are all markup and when we see the "@", we interpret "@p.Price" as an expression and stop at the ")".

So there's a quick overview of how Razor identifies and parses expressions. In my next post I'm going to discuss hosting the Razor parser outside of ASP.Net. As before, please feel free to leave comments if you have questions, or send me a tweet (@anurse) or an email (andrew AT andrewnurse DOT net).