Surface and Phoenix LiveView - what comes next?
With the recent merge of Support function components via component/3 and Introduce HTMLEngine + HEEx templates, we initiated an effort to turn Phoenix’s current template-based model into a more component-friendly approach that can drasticaly improve development experience when designing Phoenix applications. This component-based approach has been present in Surface for some time now and after a few discussions between the members of both teams, we decided to bring some of those concepts to Phoenix itself.
Before we start digging deeper into the details, I’ll try to provide some minimum context on why I decided to create Surface in the first place.
Why Surface?
The idea of components in software development is not new and its practical use has been around for at least 3 decades. And although many new concepts have been added to the original idea, they’re usually variations of the same basic principles, presented with different clothes and updated vocabulary.
Since live components were introduced a couple of years ago, LiveView users have been able to build stateful components based on
the Phoenix.LiveComponent abstraction. However, although this abstraction provided the foundation to define components that can handle
their own state, there were still many aspects that were missing when it comes to a full-featured component model.
My first attempt to use LiveView was in 2019 when I was preparing a talk on “Building Efficient Data Pipelines with Broadway” for ElixirConf. I wanted to present a live representation of the pipeline so people could visualize the workload of each stage (process) along with global and individual metrics. Basically, a dashboard for Broadway.
LiveView sounded like a perfect match for that use case and as you can see in this short video, it proved to be the right choice for the job. I was amazed that I was able to do all that stuff with absolutely no custom JS.
This first contact with LiveView led me to the following conclusion:
Phoenix Liveview is fantastic! I want it to play an important role in my dev stack. However, it needs to evolve into
a “real” component-based approach. Something similar to what React or Vue is but taking into account the server-side nature
of Phoenix LiveView. This component model should focus not only on composability but also on improving ergonomics and
dev experience in general.
In order to address this, I started to work on a prototype of what would later become the first draft of Surface. And the dashboard became the first opportunity to explore some of the ideas behind it.
What was missing in LiveView?
The main pain points I had when designing the Broadway Dashboard were mostly related to the following three gaps:
- Limited stateless component API
- No HTML/component-aware template engine
- No declarative interface for components
I’ll try to elaborate a bit on each of those gaps, presenting their direct impacts on the development experience.
Limited stateless component API
When Phoenix introduced live components, they could be either stateless and stateful. However, even though stateless components are not specific to LiveView - they are stateless after all - those components could not be used outside a LiveView, so it’s not possible to reuse them in any controller-based view nor in layouts.
In order to overcome this problem, many users have tried a more functional approach by designing those stateless pieces of code as functions instead of live components. That works perfectly until you try to reuse those functions in different scenarios. However, the issue is that those individual functions and the components themselves would often not compose. If you attempted to pass a component to a custom function, you would often see the following runtime error:
** (exit) an exception was raised:
** (ArgumentError) cannot convert component X with id nil to HTML.
A component must always be returned directly as part of a LiveView template.
If you ever tried to use a live component inside form_for, you’ve certainly seen a similar message as most of Phoenix’s built-in form/inputs
helpers rely on the contant_tag function.
No HTML/Component-aware template engine
EEx is a great template engine. It’s not only fast but also extremely flexible. The main issue with it when used as a solution
for a component-based model, is the fact that it makes no distinction between plain text and a structured format like HTML.
That’s one of the main reasons it can be so fast and flexible.
However, by not recognizing the structure of the underlying HTML template, it misses a wonderful opportunity to gather relevant information about the semantics of that structure. Information that could be used later to do amazing things that can boost productivity, like static validation, improved ergonomics and better tooling.
No declarative interface
The whole point of designing components is to provide reusable building blocks that can be easily composed into other larger reusable building blocks. In order to achieve that, we need a way to document the component’s shape, identifying its public interface from any other internal detail that should better be kept hidden from the user.
Without a standard API to declare that interface, it’s up to the component’s author to find a way to document it properly. If that does not happen, it leads to a poor experience for developers trying to use those components. Whenever you need to use a component and there’s no well-defined interface for it, you’ll have to answer the following questions by yourself:
-
which assigns are public and can (or must) be passed to
live_component/4? - what’s the type of each assign?
- which assigns are required? Will they receive default values if I don’t initialize them?
- which assigns represent the internal state of the component and shouldn’t be touched at all?
However, if we structure this information, we not only improve communication by giving precise answers to those questions, but we can also provide compile-time checking, automatic generation of docs and better tooling overall.
On the tooling front, I’d mention the ability to provide auto-complete for editors and to build tools like the Surface Catalogue, which is our attempt to bring something like Storybook to the Phoenix/LV realm. In case you haven’t seen it yet, here’s a short video of its first prototype in action.
The path to a “real” component model
The two PR’s mentioned at the beginning of this post addresses two of the three gaps listed above.
The new component/3 addresses the first one by bringing a compatible stateless component API that allows users to define real
stateless components based on pure functions. These new “Function Components” can:
- be used in controller-based dead views, including layouts
- include live components in their inner block
- be diff-tracking-aware
The second PR introduces a new templating language called HEEx, which is an extension of EEx. This language is HTML-aware and
component-friendly, providing syntactic sugar for handling attributes as well as validating the structure of the template.
Have you ever forgot to close a <div> and saw LiveView go crazy, updating parts of the view you didn’t expect? If the markup is invalid,
the browser will attempt to complete it, and it may do so incorrectly. If for any reason the structure of the HTML is broken, LiveView
will misbehave.
With the new HTMLEngine engine, users will be able to use the ~H sigil to write HEEx code directly in their
components/LiveViews or create .heex template files for them, just like it was previously done with ~L and .leex, respectively.
The engine will also validate the code, raising errors on common mistakes like those unclosed/unmatched tags.
The new syntax also allows users to inject “Function Components” directly in the template using an HTML-like notation:
<Component.func attr="value">
<div>
...
</div>
</Component.func>
An HTMLTokenizer which is used by the new engine is also available and can be used to easily implement additional
tools, like a formatter, for example. ;)
As you can see, we’re filling two of the three gaps we had. Conversations regarding the third one (No declarative interface) are already advancing.
What about Surface?
One question that has been raised in the community is if Surface will eventually get merged into LiveView. The answer is:
Not exactly. :)
Surface is still way ahead of LiveView on its component model. There are many other features and dozens of compile-time checks. We’re starting carefully to bring some of its features to Phoenix Liveview but instead of doing this indiscriminately, we’re identifying the core concepts and evaluating the best way to implement them as core features.
In the long-term, we hope we can move enough of those concepts to Phoenix, allowing Surface to evolve much faster, focusing on higher-level features, ergonomics, better tooling and high-quality components, while the Phoenix core team can keep improving the foundation of its component model.
A good example of how this is beneficial for both projects is the already mentioned component/3 macro. It would be impossible
to implement that in Surface alone as it requires changes to LiveView itself.
Conclusion
In this post, I tried to present the current efforts to push Phoenix towards a more component-friendly direction.
The end goal is to establish Phoenix as a great foundation for writing reusable components, regardless of the template engine.
If you like EEx, you’ll be able to use HEEx. If you don’t, you can use Surface or any other template language you prefer.
As long as the component model is part of the LiveView’s core, users will be able to use and share whole suites of components built
with any of those different solutions!
We still have a long way to go to achieve that but the first steps have already been taken and I hope you’re as excited as I am about the wide range of possibilities this brings to provide a modern and robust solution for web development.