Posted in: Development and Technology
Tags:

 

There comes a time in cross-platform development where you inevitably long for a simpler mechanism for dealing with the deluge of image assets. You end up with three or more sizes of every single image for every platform. At Twin, SVGs have risen to that challenge for us, and we use it everywhere we can with a custom variant of Paul Patarinski’s SvgImage control. After some minimal setup work to start using them, they render crisply at any size and resolution we throw at them.

One place that SVG use has had shortcomings for us is when we needed backgrounds with varying sizes. We need a unique SVGs for each size we intend to use because you haven’t been able to selectively stretch an SVG like you could with 9-patch images on Android or StretchableImage/CreateResizableImage on iOS. In SVG terms, this appears to be called 9-slice support, and it is something we want to both use in our apps as well as make available to you. We’re going to take a little journey through the successes, and hiccups, of bringing 9-slice support to SVGs drawn in cross-platform Xamarin.Forms apps.

Throughout this process, I’m going to be using the Twin Technologies logo because it was a simple vector I had on hand. While using a logo is not a great use of the 9-slice concept, it definitely shows what is happening in the stretching. The final result from this exploration is a demo project where you can choose an SVG and play with the insets using a slider. Feel free to clone or fork the project on GitHub and play around with the sample.

svg-9-slice-final-demo.gif

In the current demo the slider starts at 0, which causes it to fall back to rendering the SVG scaled up proportional to the output size. From there, you can drag it to change how “deep” into the image the insets go when calculating what parts of the SVG to keep statically sized and what to stretch.

Current Stretching Systems

If you aren’t familiar with stretchable images, it simply means designating a portion(s) of an image as eligible for stretching to fill in what is left after the other portions are kept at their original proportions. A common example of this is a button background. You could create a small button background and then use either 9-patch pixels or StretchableImage code to resize it to be the background of any sized button. The corners would stay crisp and the middle and sides would stretch to fill in the rest.

You start with a 9-patch PNG image like this one.

button_background.9.png

And no matter what size you render it, you end up with the same corners.

9-patch-buttons-cropped.png

The iOS equivalents, StretchableImage/CreateResizableImage, will produce similar results with offset values instead of one-pixel edges. (Do note, however, that 9-patch images can do a lot more complex things than both what StretchableImage does and what we are doing here, but those will have to be an exercise for another day.)

Breaking an SVG into Parts

There may be better ways of rendering an SVG on Xamarin, but Frank Kreuger’s NGraphics seems to be at the root of those efforts, and for good reason–it absolutely rocks for making cross-platform vector graphics. The SvgImage control mentioned above uses an older, custom, fork of the NGraphics library. Since we were already using SvgImage, I decided to point my efforts toward adding 9-slice support to that library.

In SvgImage, the core class is really just a repository for the bindable path and assembly properties. The rendering work is done in the platform SvgImageRenderer classes. The iOS and Android implementations are very similar, in OnElementChanged, the SVG is retrieved from its happy home as an embedded resource and read to get the resulting Graphic instance. This is then drawn to a platform-generated IImageCanvas instance which can generate the final UIImage or BitmapImage used on the native iOS and Android controls, respectively.

Here’s an SVG of the Twin logo rendered with just the current SvgImage system.

initial-twin-svg-rendering.png 

Rather than dig too deep into NGraphics or reinvent the rendering, my goal is to take an SVG and render the various sub-parts we need individually to a canvas the size needed for displaying to the user. To see if this works, I started small. I broke the image into quarters and tried to render it at 100 percent size with just two opposite quadrants, since rendering all four wouldn’t look any different. Here’s how that ended up after a few fumbles with SVG ViewBox use.

For any SVG section, I offset the ViewBox to the desired top-left starting point: first at 0,0 for the upper-left section, then at the center for the lower-right section. Then, I would draw it to a quarter-sized canvas that would ultimately serve to crop just the desired quadrant. As SvgImage did previously, we will extract an image from that canvas. But, instead of using that as a source for an image control, we will draw that image to a final canvas sized for display. Here is the Twin logo SVG rendered as two quadrants. (I also added a background color to finalCanvas for debugging purposes as I worked on the logic.)

quadrant-twin-svg-rendering.png

Being able to draw the same SVG with a lot more code isn’t exactly something to be too proud of, but doing nine identical slices is trivial from this same process. With some pretty background coloring on the slices for effect, here’s how that ended up looking.

all-slices-same-scale-svg-rendering.png

In this version, the inset values that determine where to divide up the SVG into slices is simply hard-coded. We’ll end up with a nice container object for these values that will contain Top, Right, Bottom, and Left inset values (all measured from their respective sides toward the inside.

inset-diagram.png

With all nine parts drawing independently, it’s just a matter of making the inner slices scale to the size that will result in a slice for the “stretched” sections in the final output. That is where ViewBox will do the heavy lifting for us.

Making an SVG “Stretch” 

The overall math for the nine SVG sections is simple. Using the inset measurements described in the diagram above, any section’s measurements from the original SVG can be determined from the width, height. And since the insets are used directly in the final output canvas to maintain the size of the corner sections, the same logic also applies to finding a section’s measurements there. For some simple examples, the rectangle that defines the upper-right section works out to this.

And the rectangle defining the center section is not much more complicated.

With a helper enum to label each slice for easier code consumption, this section rectangle calculation logic was contained in a helper function called GetSection housed within the insets struct.

If we wanted to make a stretched image 300px by 300px from our 139px wide by 79px tall Twin logo SVG with 10px insets on all sides, the resulting image would have 10px by 10px corners, and the inner pieces would scale to fill in the remaining 280 horizontal and vertical pixels. First, we will want another helper function to make an image from one of our nine slices.

Let’s dissect the parameters since they may not be complete clear out of context.

  • Graphic graphics is the original SVG as read in from its embedded resource
  • Rect sourceFrame is the frame of the slice in the original SVG (e.g., the left-center slice from the current Twin logo SVG example is 10px wide by 59px in our current example)
  • Rect outputFrame is the frame of the slice in the output image (e.g., 10px by 280px)
  • double finalScale is something specific to scaling for high-density screens on iOS via UIScreen.MainScreen.Scale

The frames passed in to this method are determined by the GetSection method described above. In order to share as much code as possible, the final RenderSectionToImage function uses a Func<Size, double, IImageCanvas> parameter passed in from the platform renderer instead of needing to call platform-specific canvas creation methods directly.

We have the basics down for translating and scaling slices, now we just need to call those methods for each slice in the enum.

This method is run from the SvgImage class, allowing it to be shared between the two renderers. Since this math and slice drawing isn’t free, when the insets are not set, we fall back to the original code and do a single-pass rendering.

Now we can really start to stretch the output of our SVG sections. In this case, that just means making the Twin logo look a little silly.

twin-logo-slice-stretching.png

But it could equally be used for rendering some fancy rounded-corner double-border button background you want to use.

test-button-streching.png

Because the draw methods for SVGs already takes screen scale into account (manually on iOS, automatically on Android), you get the same size button regardless of the device’s screen density.

What Did We Get?

With this code, you could potentially pull a working prototype system into your Xamarin.Forms projects (targeting iOS and Android) for rendering 9-slice stretched SVGs. Along the way, I moved much of the duplicated code between the platform renderers to the shared control, which is nice on its own but also should make additional implementations easier to add. (You provide a way to generate an IImageCanvas, and the rest is mostly free.) 

While setting up the shared code, not only did that reduce the code’s surface area, I was also able to set up the control to render in its draw system with invalidation based on changing properties. This means that an SvgImage can have its SVG changed at runtime (or inset values as is done with the demo project).

SvgImage is currently built on a custom NGraphics fork. As an experiment, I swapped it for the latest official NGraphics NuGet package and it worked great. Better than great, actually, since the SVG support on the main repo has become quite extensive now. I don’t know the original reasons for the fork, but here’s hoping they can reunite again.

What’s Left Here?

While we now have a working prototype of 9-Slice SVG support in SvgImage, there are definitely some refinement steps still needed before it becomes a pull request back to the original project. For one, I haven’t address Windows Phone support at all. It’s not that I don’t want to support Windows Phone; I just couldn’t test it on my current development machine, let alone a real device.

There may also some validation work needed. While having insets overlap each other isn’t technically going to blow anything up, it probably isn’t what you intend to render. In fact, if you overlap far enough, you just get a bunch of copies of your SVG.

inset-overlap-to-11.png

So, potentially one could prevent inset values from overlapping each other, but this might be easiest to just warn the user of impending craziness. If the insets are greater than the SVG width or height, though, crashes are coming for you.

Sometimes, keeping the corners the same size might not be ideal for larger output sizes. Since SVGs can happily scale, there could be potential for a CornerScale property. If you want your button to have 25px corners until it is 300px wide and 50px beyond that, there would be no reason that couldn’t be technically possible. We would just need to adapt the section rectangle math to account for inset scaling on corners.

Next Steps

The obvious next step is to tie this system into a tappable control. A quick hack to make this happen is to stack a Button over the SvgImage control within a container layout (e.g., AbsoluteLayout) and assign a Clicked handler accordingly. Ideally, we would either have a Clicked event on SvgImage or a separate SvgImageButton. Alternatively, a system to use an SVG (or 9-slice SVG) rendering could be used as an image source for something like the existing Xamarin.Forms ImageButton. 

One related issue that arose after setting up SvgImage to allow properties to change at runtime was a memory leak from string allocations out of the NGraphics SvgReader. While I could only get it to actually crash on Android, it is likely affecting iOS in ways that are just not as visible (memory pressure, GC runs, etc.). I won’t pretend to know the best way to optimize out string allocations, but hopefully there is some way to represent the SVG in memory in a reusable form without re-reading it on ever re-draw.

While this replicates the behavior of an iOS StretchableImage, it doesn’t offer the level of functionality allowed from 9-patch PNGs. All of those additional features (e.g., multiple stretch zones) could likely be hacked around with more controls, but some might be easily integrated into SvgImage directly.

Additionally, There is definitely some room for benchmarking some heavy usage of this control in a 9-slice use. If it can’t stand up the weight needed to put a stretched SVG (or multiple) per item in a large fast-scrolling list, there may need to be work to move the section drawing deeper into NGraphic’s rendering code.

The post Explorations in Cross-Platform Assets: 9-Slice Stretchable SVGs in Xamarin.Forms Apps appeared first on Twin Technologies.