Tag Archives: Open Source

Mozilla Platform Rendering in Asia

Posted on by
1

I’ve spent the last 3 weeks traveling in Asia for Mozilla and W3C business. On May 20 to 24, my team was in Taiwan for a series of meetings on Platform Rendering (Layout, Graphics, and Media.) It’s always an enlightening experience for me when I spend time with such a talented group of people:

The Mozilla Firefox Platform Rendering Team on location in Taiwan.

Going to Taiwan was a very special event. The local Taiwan office was awesome, and the local staff was very wonderful, taking great care of us while we were there. Beautiful and fast rendering requires a deep understanding of the underlying hardware, and if you’re serious about hardware, you eventually have to go to Taiwan. Meeting and working with the local Taiwan team was great. They’re so smart, and eager to learn about the Firefox Platform. We spent a lot of time getting them caught up on Rendering infrastructure, and various Mozilla-specific topics (Open Source, Open Specs, Code Review, etc.)

I’ve pasted the calendar we followed for the week, below. The links point to raw notes from the sessions (where available.) I apologize for the lack of context in some notes, as they’re meant for the attendees. They’re in this blog post to give you an idea of what the Rendering Team does when we get together in a large group every few months.

Taiwan Rendering Sessions

Monday Tuesday Wednesday Thursday Friday
9am Lightning talks Graphics: SkiaGL Canvas (snorp) Graphics:WebGL (Vlad) Layout/Media: Implementing WebVTT CSS features (rillian/dbaron)
10am Free Graphics: SkiaGL Content (Jeff M.)
Layout: Intro to Style System (dbaron)		 Graphics: WebGL (Vlad)
Media: libCubeb/Audio Latency (Kinetik)		 Platform:Cycle Collection 101 (khuey) Graphics: Timing Attacks (bjacob)
Layout:Layout Documentation – Conf Room B (jwir3)Media: ???
11am Free Graphics: Async Canvas (Vlad, snorp)
Layout: Intro to Dynamic Changes in Layout (dbaron)		 Graphics: WebGL (Vlad)
Media: Web Audio (padenot)		 Layout: CSS Masking
Media: Fixing MediaStreamGraph video propagation (roc)		 Graphics: Imagelib (joe)
Other: WebGL (Vlad)
Layout: ???
Media: ???
Noon Lunch Lunch Lunch Lunch Lunch
1pm Graphics: OMT-Compositing (Nical – “D”)
Layout: Layout 101 (Elika – “C”)		 Layout: APZC Graphics: Moz2D/Player2D (Bas) Layout:Layout fuzzer status (multicol security bugs, etc.) – Conf Rm. A Graphics:Intro to Gecko Graphics (mwoodrow)
Layout: ???
Media: ???
2pm Graphics: Layers and buffers (bjacob) Layout:Web Animations (birtles)
Media: MediaSource Extensions (kinetik)		 Graphics: OMT Canvas Layout: Web Animations contd. (birtles) Graphics: ???
Layout: Standardization and Mozilla (dbaron/abr) «IETF Slides»
Media: ???
3pm Graphics: Gralloc
Layout:CSS Variables / CSS Cascade (heycam, dbaron – “C”)		 Layout: CSS Graphics Graphics: OMT Animation
Layout: (see graphics)
Media: Media Decoding (cdouble) OMX Codec (sotaro)		 Graphics: B2G/Android Graphics Testing
Layout:CSS Writing Modes – Vertical Text (fantasai)		 Graphics: ???
Layout:Layout Performance (jet)
Media: ???
4pm Media: WebVTT/TextTrack (rillian) The Mozilla Way (roc) Graphics: CSS Filters
Layout:CSS3 Fonts		 Code Reviews – The Mozilla Way (roc) Work Week Wrap-up (jet)

Graphite Smart Fonts in Firefox

Posted on by
4

The Gecko Layout Engine added Graphite Smart Fonts support starting in Firefox 11. The Mozilla Firefox documentation for Graphite is still under construction but already contains lots of useful information for Web Designers and Developers.

The number of people who will benefit from Graphite is the small subset of the population that reads and writes with lesser-known scripts that require complex ligatures and rules for glyph substitution. In other words, we expect this work will appeal to a rather small underrepresented percentage of the web who communicate using uncommon writing systems. Firefox fully supports OpenType for the majority of writing systems on the web.

The key benefit of Graphite is that it allows type designers to build in complex script handling logic into fonts themselves, rather than relying on an application library like OpenType to explicitly implement support for it.  That’s why it’s so useful for display and layout of scripts that require features not available in OpenType.  There’s a lot of overlap with OpenType and the format itself simply adds extra data to OpenType fonts (the glyph and metric data is the same).

Graphite allows fonts to embed rich programs that implement the complex rules that govern glyph behavior in relation to other glyphs in a text run. This rich API presents challenges in an application as widely-distributed as Firefox, especially in regards to security and integration with the rest of the text subsystems.  We’re currently testing, validating and documenting the Graphite feature in Firefox. We’re interested in getting feedback from the Font Developer community to ensure that Graphite is expressive enough for complex writing systems. If you have experience with OpenType layout, we’d like to see similar layout functionality implemented using Graphite. We’re looking for examples that could include non-Latin script support (e.g. Arabic, Thai, Indic scripts) or Latin/Cyrillic/Greek fonts with extensive feature support.  A symbolic “WingDings” font with complex layout rules could also be a useful example. Please reply in the blog comments if you are interested and available to help us test and verify Graphite’s Layout API’s.

I’m posting about Graphite because it’s another example of why working on Firefox is so different from working on anything else. We don’t optimize our actions to generate maximum revenue, we do things because we want to give the entire planet equal and open access to the web. Proper text layout and rendering for complex scripts enables free expression across communities who would otherwise not have a global voice. We think it’s well worth it in service of the greater good for an Open Web.

Toggle Paint Flashing for Firefox

Posted on by
1

Paint Flashing is now in the official Release channel starting in Firefox 11.  I posted a new add-on that let’s you enable Paint Flashing with one click: https://addons.mozilla.org/en-US/firefox/addon/toggle-paint-flashing/

68828

This tool is useful for quickly inspecting how Firefox renders your web pages as it visually indicates how much screen area is being painted after the page layout is computed.The source code for the add-on is also posted.

 

 

Firefox Invalidation

Posted on by
2

As promised, I’m going to call out Firefox Layout & Rendering code changes that my team has been working on. This bug fix from Bas Schouten (as reviewed by Robert O’Callahan) was one of those light-bulb code changes that have really advanced my understanding of the Gecko Layout Engine’s graphics code:

https://bugzilla.mozilla.org/attachment.cgi?id=574190&action=diff

This check-in implements “Paint Flashing”, a diagnostic tool that shows when the browser is invalidating (or repainting) a screen region that has changed. It’s very useful for diagnosing redundant (painting too often) or greedy (painting too large an area) invalidation. Both kinds of bugs are notoriously hard to find.

Paint Flashing will paint a random color (at 20% opacity) to indicate the “damaged” area of the screen that the browser is repainting. You can enable Paint Flashing by setting this preference to true:

nglayout.debug.paint_flashing

If you’re running a DEBUG build, you can get to it in the Layout Debugger. The code is deceptively simple but shows the logical bottleneck where invalidation occurs in Firefox. We’re already using this tool as we evaluate larger invalidation changes that aim to dramatically speed up rendering in Firefox. You can also use this tool as you develop your web apps to see if performance issues are the result of invalidation problems. You can tune your app much better if you know how much screen area is getting painted from your javascript or CSS code.

Did Adobe finally kill Macromedia Flash?

Posted on by
21

Adobe announced that it is ceasing development on the Flash Player for Mobile Devices. As you may know, I was on the engineering team that brought Flash Player 10.1 to Mobile Devices back in June, 2010. This announcement is disappointing for many reasons, but not surprising given the realities of mobile web development.

While I was at Adobe, we spent a great deal of time and energy to get the Flash Player to run fast enough on the HTC Nexus One running the Android OS. At the same time, Apple was making it very clear that Flash would never be allowed to run under Mobile Safari, especially if our goal was to have it work on Android first. The writing was already on the wall even before we shipped: no matter what we did, or how well we did it, we were not going to have our ubiquitous platform extend to the Mobile world.

Of course, that’s not the whole story. What I’m truly disappointed about is that Adobe is exiting the Flash Platform business–they laid off 750 people yesterday, including the entire US-based Flash authoring tool team. This means that offering a free Flash Player runtime subsidized by selling tools is no longer a business Adobe is interested in. I always worked to ensure that SWF as a data type could have the same ubiquity as JPEG. That is, you could trust that the SWF you authored in 1998 would be rendered by a publicly available runtime in 2028. For this to happen, you need to have an organization that can shepherd the technology and commit to doing so for all time.

Unfortunately, Adobe is not that organization.

Or is it? I hope that they release the runtime code to the public domain so that others can extend and maintain it. I’m well aware that there are 3rd-party licenses in the code, but that can be easily redacted. If there is any hope left for Flash to survive, it will have to be an open-source implementation that can fulfill the promise of a ubiquitous rich-media data type for years to come. Adobe needs to acknowledge that the public needs SWF to survive, and that opening up the platform is now the only way to ensure that outcome.

Graphics Engines in Firefox

Posted on by
1

One of the key goals for the Firefox Layout & Rendering team is to improve performance for web applications. Web apps nowadays push the graphics rendering stack in ways that rival console games. To that end, our existing rasterizer was hitting its limits in terms of the sheer volume of data we now push through it on a a regular basis. Switching out a shipping application’s raster engine is a big deal that requires a lot of scaffolding. When I was working on the Flash Player, I saw how much work goes into integrating a new graphics engine into a shipping product. Sometimes, you have no choice but to tack it on top of (or in Flash’s case, behind) the existing rasterizer.

For Firefox, the scaffolding involves the addition of the new rasterizer initially for the  <canvas> API’s. This was ideal because it was fairly localized, and backwards-breakage risk was manageable. In the next phase, all CSS layout styling will use the new engine. The lessons learned from <canvas> make this a much less risky phase but it’s still fraught with peril. The benefits will be worth it, especially when we get page-draw and responsiveness metrics cranked up.

I’ll be posting up more details on Layout & Rendering internals as I dive deeper into it. It’s quite exciting to work on the guts of the Web with passionate engineers.

Shipping already? I just got here!

Posted on by
Reply

Firefox 7.0 shipped today. While I can’t take any credit for any of it, my team deserves all the praise. They’ve been hard at work on reducing memory all over the product. I’ve had a few people complain to me about “pork” in Firefox and we’re making some big changes to cut the fat. It’s interesting how getting a new job just means new complaints from the same cadre of “stakeholders.” Let me know via the blog comments if Firefox is working better (or not) with all these optimizations.