Quick Update - Microsoft WebMatrix Beta released

Scott Guthrie just announced the first beta release of Microsoft WebMatrix.  I'll leave you to check out his blog post to find out more.

This is also your first chance to try out Razor.  We haven't released the MVC View Engine for Razor, but WebMatrix includes ASP.Net Web Pages, a simple page model that uses Razor syntax.  After installing WebMatrix, just create a new site, drop a CSHTML file in it, put some code in and go!

I'll post more details later, but for now, check out the post and play with the bits!

Posted on 06 Jul 2010

Inside Razor - Part 1 - Recursive Ping-Pong

This is the first of my blog posts about the parser for the new ASP.Net Razor syntax.  We've been working on this parser for a while now, and I want to share some of how it works with my readers!

The Razor parser is very different from the existing ASPX parser.  In fact, the ASPX parser is implemented almost entirely with Regular Expressions, because it is a very simple language to parse.  The Razor parser is actually separated into three components: 1) A Markup parser which has a basic understanding of HTML syntax, 2) A Code parser which has a basic understanding of either C# or VB and 3) A central orchestrator which understands how the two mix together.  Note that when I say "basic understanding" I mean basic, we're not talking about full-fledged C# and HTML parsers here.  I've joked with people on the team that we should call them "Markup Understander" or "Code Comprehender" instead :).

So the Razor parser has three "actors": The Core Parser, the Markup Parser and the Code Parser.  All three work together to parse a Razor document.  Now, let's take a Razor file and do a full summary of the parsing procedure using these actors.  We'll use the sample that I used last time:

<ul>
@foreach(var p in Model.Products) {
<li>@p.Name ($@p.Price)</li>
}
</ul>

Ok, now we start at the top. The Razor parser is essentially in one of three states at any time during the parsing: Parsing a Markup Document, Parsing a Markup Block or Parsing a Code Block.  The first two are handled by the Markup Parser, and the last is handled by the Code Parser.  So, when the Core Parser is fired up for the first time, it calls into the Markup Parser and asks it to parse a Markup Document and return the result.  Now the parser is in the Markup Document state.  In this state, it simply scans forward to the next "@" character, it doesn't care about tags or other HTML concepts, just "@".  When it reaches an "@", it makes a decision: "Is this a switch to code, or is it an email address?"  This decision is basically done by looking just before and just after the "@" to see if they are valid email characters.  This is the default convention, but there are escape sequences to force it to be treated as a switch to code.

In this case, when we see our first "@", it is preceded by whitespace, which is not valid in an email address.  So, we now know we are switching to code.  The Markup Parser calls into the Code Parser and asks it to parse a Code Block.  A Block, in terms of the Razor Parser, is basically a single chunk of Code or Markup with a clear start and end sequence.  So, the 'foreach' statement here is an example of a Code Block.  It starts at the "f" character and ends at the "}" character.  The Code Parser knows enough about C# to know this, so it starts parsing the code.  The Code Parser does some very simple tracking of C# statements, so when it gets to the "<li>" it knows it's at the start of a C# statement.  "<li>" is not something you can put at the start of a C# statement, so the Code Parser knows that this is the start of nested Markup Block.  So, it calls back into the Markup Parser, to have it parse a block of HTML.  This creates a sort of recursive ping-pong game between the Code and Markup parsers.  We start in Markup, then call into Code, then call into Markup and so on before finally returning back up this whole chain.  At the moment, the call stack in the parser looks something like this:

  • HtmlMarkupParser.ParseDocument()
    • CSharpCodeParser.ParseBlock()
      • HtmlMarkupParser.ParseBlock()

(Obviously, I am leaving out a lot of helper methods :)).

This highlights a fundamental difference between ASPX and Razor.  In an ASPX file, you can think of Code and Markup as two parallel streams.  You write some Markup, then you jump over and write some code, then you jump back and write some Markup, and so on.  A Razor file is like a tree.  You write some Markup, and then put some Code inside that Markup, then put some Markup inside that Code, and so on.

So, we've just called into the Markup Parser to parse a block of Markup, this block starts at "<li>" and ends at the matching "</li>".  Until that matching "</li>", we won't consider the Markup Block finished.  So even if you had a "}" somewhere inside the "<li>" it wouldn't terminate the "foreach", because we haven't come far enough up the stack yet.

While parsing the "<li>", the Markup Parser sees more "@" characters, which means even more calls into the Code Parser. And so the call stack grows:

  • HtmlMarkupParser.ParseDocument()
    • CSharpCodeParser.ParseBlock()
      • HtmlMarkupParser.ParseBlock()
        • CSharpCodeParser.ParseBlock()

I'll go into detail on how these blocks are terminated later, because it is a little complicated, but eventually we finish these code blocks and we're back in the "<li>" block.  Then, we see "</li>" so we finish that block and pop back up to the "foreach" block.  The "}" terminates that block, so we back up to the top of our stack again: the Markup Document.  Then we read until the end of the file, not finding anymore "@" characters.  And we're done!  We've parsed the entire file!

I hope that's made the general structure of the parsing algorithm somewhat more clear.  The key take-away here is to avoid thinking of Code and Markup as separate streams and think of them as constructs you nest inside each other.  Our next topic will be Implicit Expressions, which is the logic that allows us to detect what parts of "@p.Name ($@p.Price)" are code, and what are markup.  I'll give you a hint, we took some inspiration from PowerShell here ;).

Please post any questions or comments in the comments section or email me at "andrew AT andrewnurse DOT net"!

Posted on 05 Jul 2010

Introducing Razor - A New View Engine for ASP.Net

UPDATE: Fixed broken examples (I hope :)).

Earlier this morning, Scott Guthrie blogged about a new View Engine we're developing for ASP.Net. As many of my readers know, I joined the ASP.Net team back in October in 2009, and I'm really excited to finally be able to share what I have been working on for the past 8 months. When I joined Microsoft, I was shown some early prototypes for this new syntax and over the course of the next 8 months, we developed it into the Beta we're going to be releasing very soon.

Writing the parser for Razor has essentially been my job for the last 8 months, so I'd like to describe a little about some of the design ideas that went in to it as well as some of the interesting ways we implemented things. This is the first in a few blog posts I'll do about the Razor syntax as well as the Parser design.

Razor syntax is designed around one primary goal: Make code and markup flow together with as little interference from control characters as possible. For example, let's take the following ASPX:

<ul>
    <% foreach(var p in Model.Products) { %>
    <li><%= p.Name %> ($<%= p.Price %>)</li>
    <% } %>
</ul>

Now, let's boil it down to the parts that we actually care about, removing all of the extra ASPX control characters:

<ul>
    foreach(var p in Model.Products) {
    <li>p.Name ($p.Price)</li>
    }
</ul>

Obviously there isn't enough data here to unambiguously determine what's code and what's markup. When we were designing Razor, we started from here and added as little as we could to make it absolutely clear what is code and what is markup. We wanted Razor pages to be Code+Markup, with a little extra stuff as possible. We even used that goal as inspiration for the file extension for C# and VB Razor pages: cshtml and vbhtml.

So, using the C# Razor syntax, the above example becomes:

<ul>
    @foreach(var p in Model.Products) {
    <li>@p.Name ($@p.Price)</li>
    }
</ul>

If you ask me, that's pretty darn close to the previous example. Razor takes advantage of a deep knowledge of C# (or VB) and HTML syntaxes to infer a lot about what you intended to write. Let's take this sample and break it down chunk by chunk to see how Razor parses this document.

<ul>

When Razor starts parsing a document anything goes until we see an "@". So this line just gets classified as Markup and we move on to the next

@foreach(var p in Model.Products) {

Here's where things get interesting. Now, Razor has found an "@". The "@" character is the magic character in Razor. One character, not 5 "<%=%>", and we let the parser figure out the rest. If you skip ahead a bit, you'll notice that there's nothing that indicates the end of the block of code (like the "%>" sequence in ASPX). Rather than having it's own syntax for delimiting code blocks, Razor tries to add as little as possible and just uses the syntax of the underlying language to determine when the code block is finished. In this case, Razor knows that a C# foreach statement is contained within "{" and "}" characters, so when you reach the end of the foreach block, it will go back to markup

<li>@p.Name ($@p.Price)</li>

Now, things get even more interesting. Didn't I just say we were in Code until the ending of the foreach? This looks a lot like Markup, and we're still inside the foreach! This is another case where Razor is using the syntax of the underlying language to infer your intent. We know that after the "{" C# is expecting some kind of statement. But, instead of a statement, we see an HTML tag "<li>", so Razor infers that you intended to switch back to Markup. So we've essentially got a stack of 3 contexts: When we started out we were in Markup, then we saw @foreach so we went to Code, now we've see <li> so were back in Markup. At the closing </li> tag, we know you've finished the inner Markup block, so we go back to the body of the foreach.

}

Then we see the end of the foreach block, so we go back to the top-level Markup context.

</ul>

And we continue parsing markup until the next "@", or the end of the file. You may have noticed I skipped over a bit in the middle of the <li> tag. I'll save the details of that for my next post, but the essential logic is the same: "@" starts code, and we use C# syntax to tell us when that code block is finished.

It's great to finally be able to share the result of our hard work with everyone. We've worked really hard to try to create a really clean syntax for mixing code and markup and I know I'd love to hear your feedback. Post in the comments on my blog, send me a tweet at "@anurse" or send me email at "andrew AT andrewnurse DOT net".

And I know you're all probably eagerly awaiting a chance to try this out. Don't worry, we'll have a public beta soon that you can try out!

Posted on 03 Jul 2010

TechEd North America 2010 - Day 1

Day 1 marked the official start of TechEd North America. And while everyone else was in the keynote, I was scrambling slightly as I realized that our demo machine was completely empty, just a clean install of Windows 7 and Office 2010. Fortunately, Brad Wilson came to the rescue, as he had been given a USB key with Visual Studio 2010, our demos and our rotating PowerPoint deck on it. We snapped it in and got started setting up.

We took advantage of our large-ish screen and internet connection to stream the keynote live, even though it was happening right next door (though given the size of the exhibition hall, right next door is a long way away :)). At the end of the keynote, the great voice in the sky (aka the announcer) told us the expo was open, and in came the swarms of people! Well, kinda. Of course, there were three "foundational sessions" (like mini-keynotes on more specific topics, like Developer Tools, etc.) going on soon after the keynote, and many people had sessions to attend, so it was pretty quiet in that first shift.

At 12:30, when my shift ended, Brad and I and a couple of people he knew from Patterns & Practices wandered over to the Meal Hall (Which is no small feat... the New Orleans Convention Center is 1.5 miles long! Granted, we're only using 1 mile of it :)). There we ate "food" (convention food is a close approximation of food in most cases... it was edible and likely had some nutrients in it... close enough) and then I trekked off to my first (and only) session of the day. Don't get too excited, I wasn't speaking, just attending. No talks for me this time around, but I do plan to get into speaking soon. Also, it is notoriously difficult to get speaking slots at TechEd if you haven't given a TechEd talk before.

I attended a talk on Windows Phone 7, which included a pretty good demo. Aside from the demo it was mostly things I knew about already. By the way, all the talks are going to be posted for FREE viewing by the general public on TechEd Online (http://www.msteched.com/), this talk was "WPH201 - Windows Phone 7: A New Kind of Phone"?. It was only slightly ironic that every once and a while, during the talk, I popped open my iPhone to check the latest news from WWDC :). Still, WP7 looks nice, and I'm basically decided that my next phone will likely not be an Apple phone. It may be a Windows Phone or an Android Phone, but even after the iPhone 4 announcement, I'm not really convinced... yet. I'll save that rant for later though :).

That evening, the exhibition hall was open for a reception at which free food and beverages (read: alcohol) was offered to all attendees, while they wandered around the various booths from Microsoft and third-party Vendors. It was basically all-hands-on-deck for that event, and we were definitely pretty busy. A number of great questions from customers of all knowledge levels, from someone who was just getting back in to Web Development and had no idea what had been going on in ASP.Net, to some folks who used MVC and wanted to know about MVC 2.

After that, I basically just wandered back to my hotel and crashed. It was a long day, and I had another one ahead of me!

Posted on 08 Jun 2010

TechEd North America 2010 - Days 1 and 0

I know, it's been many many months since I blogged. I wish I could say it was because I was really busy, still getting settled in Seattle, or because I was working on lots of cool stuff I can't talk about yet. While those are all true (*wink*), the real reason is that I just got a bit lazy with the blogging. Anyway, whirlwind update: I've been working on the ASP.Net team full-time for 8 months now, and I'm working on some cool stuff that I'll be able to talk more about soon.

In the mean time, I can talk about the fact that I'm down in New Orleans right now, working as Staff for the ASP.Net booth at TechEd North America! I'm going to try to blog a bit about the conference, starting with my first three days.

I left on Saturday, two days before the conference actually started (hence "Day -1"?). After my wonderful girlfriend woke up early to drive me to SeaTac airport at 5am for my 7am flight, I was on my way to New Orleans via Houston. The landing in New Orleans was definitely the roughest landing I've ever been in (second roughest: major cross-winds in a Dash-8 on final approach into Ottawa airport). Upon walking off the plane, I felt like I had just walked into a vat of pea soup. Having never really travelled to this part of the continent before, I was unfamiliar with the intense humidly (though I had been well warned).

After a short taxi ride, I reached my hotel, The Hotel New Orleans, conveniently located right next to the Convention Center (thus limiting my time in the un-air-conditioned air). It's a nice hotel, with a cool inner atrium design:

IMG_0314IMG_0310 IMG_0311

After checking-in, I wandered over to Canal St. and the outskirts of the French Quarter

IMG_0308

I made my way back to my hotel and decided to eat in the hotel restaurant. I had been given a list of a few things to try while I was here, one of which was Catfish, so I decided to give it a go. After eating that and washing it down with a glass of Sweet Tea (which is a very descriptive name, and a very tasty drink) I basically just crashed early.

The next day, Sunday, was also pretty relaxed. After a long sleep-in, I wandered over to the Convention Center to register

IMG_0321

My only commitment was the staff orientation at 3pm, where we got to check out our booth, located in the Technical Learning Center, which is basically where all the Microsoft teams set up booths.

IMG_0315 IMG_0316 IMG_0317

Past TechEd attendees may remember that in previous events, this was a separate section from the Partner Expo (where all the third-party vendors set up). In 2010, the TLC and the Partner Expo are all in one big Exhibition Hall. After sizing up our booth and meeting the team, I wandered to a nearby restaurant for a Shrimp Po'boy and Hush Puppies (yet more things on my list of foods to try). I also wandered along the New Orleans Riverwalk and snapped a few pics of the Mississippi (which almost form a panorama!)

 IMG_0319 IMG_0320 IMG_0318

Then, I crashed again, this time setting an alarm so I could be up in plenty of time for my duties the next day.

Next up: Day 1 - The first actual day of the conference!

Posted on 07 Jun 2010