Jekyll2021-06-25T03:18:58+00:00https://calebmer.com/feed.xmlCode → SoftwareCaleb Meredith’s blog.The continual evolution of Airtable’s codebase: Migrating a million lines of code to TypeScript2021-04-26T00:00:00+00:002021-04-26T00:00:00+00:00https://calebmer.com/2021/04/26/the-continual-evolution-of-airtables-codebaseHow to survive (and thrive!) in software engineering interviews2020-07-12T02:00:00+00:002020-07-12T02:00:00+00:00https://calebmer.com/2020/07/12/how-to-survive-in-software-engineering-interviews<p><img src="/assets/images/posts/2020-07-11-how-to-survive-in-software-engineering-interviews/interviews.jpg" alt="Cute cartoon of an interview" /></p>
<p>I’m an engineer who’s beginning to interview candidates for coding work. I’m loving meeting new people, and I’m also benefitting a lot from the incredibly interesting conversations that spark in the interview room. However, while it’s exciting for me to see the process from the other side, I know full well how daunting it is to be in the interview chair. I wrote this article hoping to assuage some fears by removing some of the mysteries around interviewing for software engineering positions.</p>
<p>Right now, my understanding of the gap between being an interviewee and an interviewer is developing. As I learn more about the process from the interviewer side, from colleagues who’ve been interviewing for far longer than I have, I can see clearly (with hindsight) some of the more egregious errors I made in the interviewee chair. I could tell stories about the things that I’ve done…</p>
<p>Ok, just a few. Purely for illustrative purposes. And maybe some amusement. Let’s call these—uhh—Story Charmander, Story Squirtle and Story Bulbasaur. Because why not?</p>
<p><strong>Story Charmander</strong></p>
<p>The worst interview I ever had ended early. Why? I was way too forthright with my opinions, and immovable on my stance. Things I’m still working on today. And while I am a huge proponent of confidence (see below), my nerves had pushed me right through confidence and into arrogance. Not a good interviewing look! More on this later.</p>
<p><strong>Story Squirtle</strong></p>
<p>My second story is about a time when I couldn’t come up with a simple answer, so instead grasped at the most esoteric, niche data structures I could muster (<a href="http://lampwww.epfl.ch/papers/idealhashtrees.pdf">Bagwell’s ideal hash trees</a> with a branching factor of 32, since you asked) to solve…wait for it…a problem asking me to do date math skipping weekends. The interviewer was, to their eternal credit, extremely patient, and once I’d stopped grasping at straws, gently queried whether that would really be the most efficient, cost effective solution for a client. That brought me back down to earth pretty quickly.</p>
<p><strong>Story Bulbasaur</strong></p>
<p>The third story is about the time I helped build an app and coded in right / left swiping with <a href="https://redux-saga.js.org/">redux-saga</a> that almost blew my own mind with its elegant beauty. Yes, it was that good. I wanted to explain all the awesome details of the code to my employer. And my employer <del>was so fascinated that he paid me double to write a book about it</del> didn’t care. Not one single bit.</p>
<p>I still think fondly about that code, from time to time.</p>
<h2 id="so-youre-preparing-for-a-software-interview-welcome">So, you’re preparing for a software interview? Welcome!</h2>
<p>Now that I’m in the other chair, I find that I have empathy with both sides. One of the really interesting things I didn’t appreciate before is that the interviewing process really is all about communication. By which I definitely mean two-sided communication. This train of thought got me mulling on all the things that I wish someone had told me before I started interviewing. So here, I’m going to explain some of the things that we look for in candidates. My intention is to make the process less intimidating, if possible.</p>
<p>There are a lot of myths around; stories that seem designed to worry candidates. I was certainly anxious, particularly given my oddly shaped background. And this needless worrying did me no favours at all. This article is intended to be more transparent, dispel some of those myths, and explain the process a little better.</p>
<p>I also want to spend a little time exploring how personal behaviors that we put on for one reason can very create different impressions on those around us (please refer back to Story Charmander). I hope that the following points will remove some of the mystery surrounding interview rooms, and maybe even allow you to enjoy the experience of meeting people who, most likely, share some of your passions with you.</p>
<h2 id="interview-terrors-taming-the-beast">Interview terrors: taming the beast</h2>
<p>Lots of people get these. Arguably most people. We all feel worried, agitated, like our stomachs are full of <del>butterflies</del> raging, fanged netherbeings.</p>
<p>It doesn’t have to be this way! Having some nerves before important events is a good thing, causing adrenaline to flow, which in turn will help your natural charisma to shine in the room. However, unchanneled fear is more harmful than not, in several ways. Here, I’m going to share with you a few strategies I’ve learned to face some common pitfalls within engineering interviews.</p>
<h2 id="rejections-fresh-rejections-get-em-while-theyre-hot">Rejections! Fresh rejections! Get ‘em while they’re hot!</h2>
<p>Some people hamstring themselves, being so worried about failing the interview that they fail to even apply for the job. This is a huge shame. Really. I almost want to shake some of my intelligent, awesome friends when I think about the jobs that they’ve settled for, instead of applying for the jobs they deserve, all because of a lack of confidence. Their inner voice made the decision that they weren’t going to get the job, and they then made sure of that by never applying.</p>
<p>Look, truthfully, for most people, the fact is that you will apply for more jobs than you land, i.e. you will face some rejections. This is ok! Please, don’t let fear of rejection put you off applying in the first place. If you don’t give yourself a chance, why would someone else?</p>
<p>I think we’re all familiar with <a href="https://www.entrepreneur.com/article/311319">rejection-to-success stories</a>; everyone has at least one and you can fall down a plethora of internet wormholes reading them. So I won’t repeat those stories here. However, recently, I‘ve been finding it more helpful to train myself to look forward to the rejections — to consider each rejection as one stepping stone closer to a YES. I’m really enjoying <a href="https://www.rejectiontherapy.com/about-jia">Jia Jiang’s rejection therapy</a> talks as a way of understanding this process.</p>
<p>I like to imagine the process as collecting a bunch of rejections that I can then trade in for a YES at a swap meet. You may have a different visualization technique.</p>
<h2 id="the-interview-itself-shall-i-compare-thee-to-a-band-aid">The interview itself: Shall I compare thee to a band-aid?</h2>
<p>Ah, the best bit. Are you a ripper-offer, or a slow peeler?</p>
<p>In theory, a perfect interview would be you demonstrating your smart, cool, funny qualities so well that when you leave, the interviewer(s) can’t imagine being on a team without you. Perhaps you’d have a conversation ignited by glowing sparks about developments and solutions in a field you’re both interested in. (Hopefully. If the interests aren’t shared, there may be bigger problems to fix 😉).</p>
<p>However, in practice, know this: the interviewer is looking for indications of your skills and behaviors. Essentially, they want to know three things:</p>
<ol>
<li>Can you do the job as advertised?</li>
<li>Can you learn fast?</li>
<li>Are you a person they want to work with for the next three—five years?</li>
</ol>
<p>And don’t forget, crucially, the interviewer should also want to show <em>you</em> how cool the company is to work for.</p>
<p>Anything else is just icing on the cake. Flowing conversation would be amazing, but honestly, one interviewer may be trying to remember to assess you and not just have a fun conversation, another may be worried about ensuring you’re having a good time; meanwhile you are trying to remember how html works. No one single interviewer gets a final say in these things. What I’m trying to say is that it’s tough, and there are a multitude of moving parts to interviews.</p>
<p>And recently, I’ve actually been thinking, instead of sugar coating this process, maybe <em>the truth</em> is really the best that can be said about this. An interview isn’t intended to be a walk in the park. Hey, 👁👄👁. Just acknowledging that fact will allow you to get to grips with the nature of the beast.</p>
<p>So yes, you <em>are</em> being evaluated while you are in the room. But please don’t forget, you should be doing some evaluating of your own while you’re in the room! Do you want to work with your interviewers? Do you get a good vibe as you walk through the offices? Interviews are not one-sided lectures—you should actively participate in making your own judgments about the company. Understand that the interview is a necessary part of this process. It might not be pleasant, but facing it with strength allows you to have some power over the situation.</p>
<p>Also please note, if you’re more of a slow band-aid peeler, sorry—I don’t know of other, more useful advice for you.</p>
<h2 id="the-interview-itself---some-answers-are-more-right-than-others">The interview itself - some answers are more right than others</h2>
<p>Let’s get into the details. The list of three things above is a summary, and there is a bunch of information out there on <a href="https://www.indeed.com/career-advice/interviewing/interview-question-tell-me-about-yourself">learn how to talk about yourself, your achievements and your abilities</a>. I have faith in you—you’ve got this part.</p>
<p>But that stuff? That’s the basic stuff. Here’s where you’re really going to make an impression. Try to remember at all times that you are being considered not just for the job as described, but as an individual who’s going to be able to grow and develop alongside the rest of the people in the company. Your interviewers want to see that you have that potential. How do we look for it?</p>
<p>Well, there are a few indications, and one really interesting one I want to talk about relates to Story Bulbasaur (see above).</p>
<p>The moral of that cautionary tale is that yes, code can be beautiful, and complex, and imaginative. Is this always what a client is paying you to do? No. Is it ever? Unlikely. Your clients probably won’t care about the intricate, clever solution you came up with to fix their issue. They just want to know it’s fixed.</p>
<p>But wait! There’s more … On the other hand, your co-workers will care <em>very much</em> what solutions you are using, as they’ll be the ones helping to maintain and develop your code in the future. They will also be the audience you want to woo with well-designed code.</p>
<p>So long story short, in the interview room, when we are looking for potential, we are almost certainly looking for your ability to take a practical approach when it comes to work. The trick to this is to try to achieve a balance between demonstrating your knowledge, and being pragmatic.</p>
<h2 id="the-interviewers-justsome-other-people">The interviewers: just…some other people?</h2>
<p>A point so important, so vital, so often-forgotten: we’re just people. Moreover, we are people who want to stop interviewing other people so we can get back to work, so that we can go home on time. This incentivizes us to be on your side — truly, we want you to be ✨The One✨ so that we can end the interview early!</p>
<h2 id="techniques-for-improvement">Techniques for Improvement</h2>
<p>So, now you have some of the common pitfalls and experiences that many of us go through on this journey. Here, if you’re interested, are some of my thoughts on how to change your approach to software interviews from now on.</p>
<h2 id="confidence-is-key">Confidence is Key</h2>
<p>Work on your confidence before getting into the room. Really work on it. Use affirmations or whatever is necessary. You need to be able to talk with assurance about what you’ve achieved, and use your skills to solve problems while in the room. If you don’t believe in yourself, it’s going to be harder for you to do these things. The interview is about proving to us in person that you have the skills your resume said you had.</p>
<p>Always try to aim for balance. At this point, please refer back to my personal ‘Story Charmander’. For some people (guilty), confidence under pressure has a habit of mutating into something that seems like arrogance. Work on your communication skills before the interview with friends and family members. Remember, we can’t read your mind, so something that may seem obvious to you will need to be explained to people who don’t live inside your head. Think of it like this: remember how for school math tests, you couldn’t get high marks unless you showed your work? Show us your work! We want to know how you’re arriving at your conclusions.</p>
<p>And finally, take the time to understand that in this area, just like everything else in life, there are very few cut-and-dry answers. Almost everything will have grey areas which can be debated and played with. Part of any interviewer’s job will be to test your boundaries and see how well you work in a team. As long as you can’t be malleable and accept other points of view, you’ll be telling your own ‘Story Charmander’ to people for a while.</p>
<h2 id="practice-may-not-make-perfect-but-it-sure-builds-muscle-memory">Practice may not make perfect, but it sure builds muscle memory</h2>
<p>I wholeheartedly recommend that you practice interviewing in preparation for an interview. Find a friend who is familiar with interviewing and set up some Zoom time with them. Or join an interview practice group - these do exist, and you just have to find them. You’re not practising to get perfect, because you won’t. What you will do is improve and build your personal tool-set. Work on spotting patterns rather than aiming to do every problem under the sun. We’re interested in the solutions you offer, sure, but also your routes to solutions, the way you think, and the tools you have at your disposal and choose to use.</p>
<h2 id="programming-languages-and-problem-solving">Programming languages and problem solving</h2>
<p>Try to be comfortable in at least one programming language, but don’t get hung up on your usage speed. Remember, we’re not looking at how fast you can type, or how many languages you can list. We are looking at how you use the tools at your disposal to effectively solve practical problems. Often, a candidate who is very familiar with only one language but has spent time practically using a language will outperform a candidate who is familiar with three—four languages but spent no time practically using those languages.</p>
<h2 id="will-i-have-to-use-a-whiteboard-like-on-tv">Will I have to use a whiteboard like on TV?</h2>
<p>Most likely yes, this will probably come up. You may need to show your work on a whiteboard. Seems like it’s easy as pie to simultaneously think, talk, write on a board and explain your thinking when the people on TV do it, right?</p>
<p>Yeah, it’s not.</p>
<p>It requires some serious hand-eye co-ordination, the ability to organize your thoughts, and simultaneously present. It’s further complicated in current, social distancing times with the necessity to ensure that:</p>
<ol>
<li>Whatever program you’re using for the interview will allow you to present; and</li>
<li>You have enabled the necessary permissions to be able to present.</li>
</ol>
<p>My advice?</p>
<p>Practise presentation skills with a friend who enjoys being talked at. Use <a href="https://www.google.com/docs/about/">Google Docs</a> and <a href="https://miro.com/">Miro</a>. Check your settings several times before any interviews begin. Then practise presenting again.</p>
<h2 id="conclusion">Conclusion</h2>
<p>The thing I most want you to take away from this is the fact that we are all just people. If you and we are in a room together, it means the recruiting team believes you can pass based on your resumé. Whether your resume is full or not, you’re in the room, so try to have fun talking about the thing you enjoy doing! And know this:</p>
<p>We, as interviewers, want you to succeed so that we can go write some more cool stuff. We want you to succeed so that we can have an extra person on the team to do even more cool stuff with. The interview is simply an opportunity for you to wow us in person.</p>
<p>Now, go forth and interview.</p>
<p>And let me know if you have any questions.</p>
<p>Because I’m interested.</p>T Shaped People2020-06-18T00:00:00+00:002020-06-18T00:00:00+00:00https://calebmer.com/2020/06/18/t-shaped-people<p><img src="/assets/images/posts/2020-06-18-t-shaped-people/cover.jpeg" alt="Person with arms outstretched in a T shape on a hill overlooking a lake" /></p>
<p>As I get the hang of interviewing software engineers, I’m learning a lot from my colleagues who have been doing it much longer than me. One of the things that’s most surprising to me is that they don’t look for exact skill set matches for the interview. As we all know, software interview questions aren’t the best indicator for what day-to-day working with a person is like. To answer perfectly, the questions we ask demand a really broad set of technical expertise. Most candidates that we see (at least at more junior levels) simply won’t have that broad expertise yet. Here’s the cool thing though - we’re not looking for a perfect answer! Instead, we’re reviewing the candidate’s problem-solving approach, and checking the tools they have at their disposal to solve the problem. We do this a lot of the time by looking for a depth of knowledge in some area of technical focus, complemented by a breadth of knowledge to passably solve related problems.</p>
<p>This reminded me of an idea from pop-psychology: the <a href="https://chiefexecutive.net/ideo-ceo-tim-brown-t-shaped-stars-the-backbone-of-ideoaes-collaborative-culture__trashed/">T-shaped person</a> (not to be confused with people practising <a href="https://en.wikipedia.org/wiki/Flag_semaphore">Flag Semaphore</a>, btw). The idea describes someone who has deep expertise on at least one topic they can uniquely contribute to the team. For example, they might be really good at performance instrumentation or visual design or data analysis. That’s in addition to the person having a range of knowledge on other topics relevant to the team so they can be an effective collaborator.</p>
<p>Although informal, the term ‘<em>T-shaped person’</em> neatly encompasses two things: how software engineering teams are approaching their hiring processes, and the growing shift towards more holistically optimised workplaces (a fancy way of referring to workplaces designed not only for efficient work, but also for nurturing and encouraging the humans responsible for that work). Having T-shaped people around isn’t always about yoga. It does two big things. First, it promotes a focus on specialization, and specialization really helps organizations of people do great things. Each person can focus on their unique area in tandem with other team-mates focussing on theirs, and the whole becomes greater than the sum of its parts. Just like in sports teams—<em>Together, Everyone Achieves More</em>. This is true both within your company’s software engineering department, and within your company as a whole. Second, the T-shaped person concept is distinguished from specialization alone by a separate, important emphasis on breadth. Great teamwork results when all members also prioritise knowing enough about everyone else’s areas to collaborate effectively.</p>
<h2 id="so-how-do-i-become-a-t-shaped-person">So, how do I become a T-shaped person?</h2>
<p>To grow as a T-shaped person there are several things you can think about doing:</p>
<p><strong><em>Deepen your focus areas of deep expertise</em></strong></p>
<p>Whatever your specialized area is, get better at it. All the greats, in whatever field, will say that the learning never stops. Satya Nadella talked recently about <a href="https://www.inc.com/justin-bariso/microsofts-ceo-just-gave-some-brilliant-career-advice-here-it-is-in-one-sentence.html">‘learn-it-all’ individuals ultimately doing better than ‘know-it-all’ people</a>, in parallel with the growth mindset being emphasised in tech companies right now. Stay curious, stay hungry and go deeper into your chosen topics.</p>
<p><strong><em>Go wide; expand your knowledge base into areas that are particularly valuable within your team.</em></strong></p>
<p>Strive to have a better understanding of at least the basics of what other people on your team specialise in. This is the route to great collaboration. Makinde Adeagbo wrote a powerful piece <a href="https://blog.devcolor.org/what-the-top-tech-companies-are-looking-for-in-engineers-9a6b4bcc91aa">here about the need to be able to handle change within tech</a>, tech being the cutting-edge industry that it is. It’s not only possible but likely that the best apps / language / tool at the moment may not even exist five years from now. This requires you to be adaptable, with the mental agility to change direction and handle uncertainty.</p>
<p><strong><em>Push at your boundaries, test your limits</em></strong></p>
<p>This one has wider benefits than just for work. Read and absorb everything that interests you, no matter what the topic. Are you in finance and interested in soccer? Go nuts with fantasy football and leagues. Get your Thursday practise in. It’s been proven many times over that <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2863117/">using your mind and body in different ways from normal work improves mental and physical functionality overall</a>. So while it might not be intuitive, focussing on post-work soccer might just clear your mind enough to help you complete that complicated debugging.</p>
<p><strong><em>Take your side project seriously</em></strong></p>
<p>A lot of people talk about having hobbies as if they’re just ways to pass time outside of work or sleep. Change this! Set yourself real goals outside of work. If you’re a developer who loves dancing, figure out how to take part in dance competitions, and sign up. I’m not kidding. I have engineer co-workers who are semi-professional hip-hop and salsa dancers. A friend of mine wrote a novel by the side of his work. I work with a lawyer who’s also a professional actor. I could go on, but my basic point is this: stretching yourself past your comfort zone, outside of your area of expertise, forces you to reach new heights and ensures you’re utilising that vital <a href="https://en.wikipedia.org/wiki/Neuroplasticity">mental plasticity</a>. It’ll all help you get better at your <a href="https://docs.google.com/document/d/1ELzlBJqg5IlsWo56JYo7af4sAMEDxabVbe8NtBAWtqA/edit">real job and find your flow, honest</a>. Plus you’ll probably enjoy life a little more.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Bear in mind, having and displaying a T-shaped skill set is not the be-all and end-all of how to succeed in a software engineering interview. It’s just one, slightly simplistic way to describe a solid set of attributes that will serve you well. As an interviewer, I see that people who have these attributes (or are working on developing them), are more able to deal with situations that occur in a real-life workplace. My conclusion is that they’re good characteristics to have!</p>
<p>And here’s the cool bit: a lot of becoming a T-shaped person is within everyone’s reach! It’s just habit formation. You need to make a habit of being curious, of reading and imbibing all the experiences you can.</p>
<p>Simples.</p>Learning programming during virus season2020-06-08T00:00:00+00:002020-06-08T00:00:00+00:00https://calebmer.com/2020/06/08/learning-programming-during-virus-seasonFive Programming Languages for 20202020-05-31T00:00:00+00:002020-05-31T00:00:00+00:00https://calebmer.com/2020/05/31/five-programming-languages-for-2020Tired: Golf, Wired: Boardgames2019-07-15T19:00:00+00:002019-07-15T19:00:00+00:00https://calebmer.com/2019/07/15/tired-golf-wired-boardgames<p>My mom is an accountant and a lot of people in her office enjoy playing golf.
However, my mom doesn’t play golf so that means to join recreational business
outings that involve golf she’s either aimlessly wandering the playing field or
not participating at all.</p>
<p>This got me thinking, what is the future of business recreation? Will it
continue to be dominated by golf, or is golf as business recreation a
generational thing?</p>
<p>I predict that the future of business recreation in the next 20–50 years will be
boardgames! Boardgames are great, they are:</p>
<ul>
<li><strong>Cheap:</strong> The most expensive games max out around $60. That’s quite
reasonable when you have 5–8 players.</li>
<li><strong>Inclusive:</strong> Anyone can play! Generally, everyone will be around the same
experience level. These games are also designed to be fun for newcomers.</li>
<li><strong>Collaborative:</strong> A good game will also serve as a team-bonding exercise. You
can get to know your co-workers better by playing a co-operative game or by
playing against each other in a hidden information game.</li>
<li><strong>Short:</strong> You can pick a game that fits into the time you have. Games with
quick 30 minute rounds allows parents to play while still getting home to
their families.</li>
<li><strong>Flexible:</strong> All you need is a table, people, and the game. You don’t need a
full golf course or equipment.</li>
</ul>
<p>I’m not talking about classic boardgames like Monopoly or Risk. I’m talking
about the <em>huge</em> wave of indie boardgames.</p>
<p><img src="/assets/images/tired-golf-wired-boardgames/boardgames.jpg" alt="Box art of various boardgames" /></p>
<p>If I’m right, and boardgames become a mainstream part of business recreation,
then the ability to run a boardgame night and introduce new players to the
activity will be pretty valuable.</p>
<p>In this post I’ll share some of my boardgame recommendations, specifically ones
I think would work well in the workplace. I haven’t played all of these…yet. If
you’re in the Bay Area and you’re interested in the future of “business
recreation” we can give one of these a spin.</p>
<h3 id="one-night-ultimate-werewolf">One Night Ultimate Werewolf</h3>
<p><strong>Time:</strong> 7 minutes, <strong>Difficulty:</strong> easy, but can become more challenging as
you add roles</p>
<p>My first, and strongest, recommendation is
<a href="https://www.amazon.com/dp/B00HS7GG5G/ref=cm_sw_em_r_mt_dp_U_uNohDb2T9JTWE">One Night Ultimate Werewolf</a>.
The name is a bit of a mouthful, but all the words actually mean something. I’ll
call it “Werewolf” from here on out.</p>
<p>In Werewolf you have three to eight players and you give everyone a role. Two or
so players will get the “werewolf” role and everyone else will be human
villagers. The humans have five minutes to correctly identify the werewolves. If
the humans discover the werewolves, humans win. Otherwise the werewolves win. So
the werewolves must lie and pretend they are villagers to not get caught. The
game also gives some human roles “powers” like the power to see the role of one
other person or the power to swap two roles.
<a href="https://youtu.be/bJ4Hrp8gQ-E">You can watch some game play here.</a></p>
<p>Werewolf is in the hidden information genre and is actually a very common format
for tabletop games. Growing up, I knew this format as
“<a href="https://en.wikipedia.org/wiki/Mafia_(party_game)">Mafia</a>” which is a game
you play with a standard deck of 52 playing cards.</p>
<p>However, Werewolf makes a couple big changes which streamlines the format and
makes it much more fun.</p>
<ul>
<li>An entire game of Werewolf fits into a single round. Hence why “One Night” is
in the title. Normally, in a Mafia-style game you have multiple rounds where
players are progressively eliminated. In a one night format players don’t get
eliminated so no one has to stop playing!</li>
<li>Furthermore, you don’t need a game master! Normally, Mafia requires one person
to run the game to make sure no one is cheating. Werewolf instead comes with
an app which will run the game for you. That way no one has to sit out during
play.</li>
</ul>
<p>These two changes mean everyone in a business recreation setting can play and
can keep playing for multiple rounds. A Werewolf round is also really short
which allows for quick games.</p>
<p>Werewolf is a very collaborative game that’s great for team bonding. You really
need to know the people around you at the table. To know who your allies are and
to know when someone is bluffing. I strongly recommend Werewolf for business
recreation.</p>
<h3 id="sheriff-of-nottingham">Sheriff of Nottingham</h3>
<p><strong>Time:</strong> 1 hour, <strong>Difficulty:</strong> medium</p>
<p>In
<a href="https://www.amazon.com/dp/B007EZMABG/ref=cm_sw_em_r_mt_dp_U_OkphDbWQ2YF59">Sheriff of Nottingham</a>
every player is a merchant trying to stock their shop with goods. One player is
also the sheriff which rotates to someone else every round. In a round, players
will put some goods into a pouch. For example, three apples. However, players
may also smuggle contraband, like beer, into their pouch! Each player gives
their pouch to the current sheriff and they declare what’s in the pouch. The
sheriff may choose to either believe the player or inspect the player’s pouch.
If there is no contraband in the pouch, the sheriff is fined. If there is
contraband in the pouch, the player who tried to smuggle is fined. Of course,
you can also bribe the sheriff to not inspect your pouch.</p>
<p>If you’ve every played the game
<a href="https://en.wikipedia.org/wiki/Cheat_(game)">Cheat (aka BS)</a> with a standard
deck of 52 playing cards, then you get the basic idea. You want to smuggle
contraband past the sheriff since contraband is valuable, but you also don’t
want to get caught.</p>
<p>It can take a second to understand all the rules. Make sure one person reads the
rule book ahead of time if you’re in a group of new players.
<a href="https://youtu.be/Gli9C3HtF44">You can watch some game play here.</a></p>
<p>The physical elements of the game are very engaging. Whether you’re putting
cards into a pouch and sliding it over to the sheriff, tossing coins over in a
bribe, or popping open the button on a pouch looking for contraband gives an
authentic smuggling experience.</p>
<h3 id="captain-sonar">Captain Sonar</h3>
<p><strong>Time:</strong> 1 hour, <strong>Difficulty:</strong> medium</p>
<p><em>Disclaimer: I haven’t played this game.</em></p>
<p>In
<a href="https://www.amazon.com/dp/B01EZUCHOC/ref=cm_sw_em_r_mt_dp_U_wwphDb18EETTX">Captain Sonar</a>
you have two teams made of two to four players with each team piloting a
submarine. Players have roles like navigator, engineer, and sonar operator to
position their submarine against the opponent. The navigator calls out the
direction the submarine is moving on a grid. The enemy sonar operator is
responsible for listening to the opposing captain to record enemy movements.
Once one team believes they have the location of another team they may fire a
torpedo.</p>
<p>So this game is a lot like Battleship except each team only has one submarine
and that submarine is piloted by multiple players.</p>
<p>This game is all about teamwork and communication. With good communication you
can strategically manuever your submarine, with bad communication your opponent
will know exactly what you’re up to. The game can get hectic with a lot of
people talking at once, but that’s also part of the thrill.</p>
<h3 id="dominion">Dominion</h3>
<p><strong>Time:</strong> 30 minutes, <strong>Difficulty:</strong> hard</p>
<p><a href="https://www.amazon.com/dp/B01LYLIS2U/ref=cm_sw_em_r_mt_dp_U_OcqhDbAG7WN2S">Dominion</a>
is a deck building game for two to four players. You start with a couple cards
and you slowly buy more, building up your deck. Dominion is a lot like Magic the
Gathering, in fact, it’s frequently marketed with: “Want to play Magic the
Gathering but don’t have the time or money to invest in Magic? Play Dominion
instead!” Which is perfect for me. I love deck building games, but the Magic
investment always intimidated me.</p>
<p>Dominion comes with a bunch of different kinds of cards. You play each game with
a different set of those cards. That means you can have multiple games of
Dominion with <em>wildly</em> different feels depending on the cards you play with.</p>
<p>The game is pretty easy to learn, but there’s a lot of depth, complexity, and
strategy to the game. Especially when you add in expansions. A player can easily
pull ahead of their competitors by employing strategy. This game might be a bit
less inclusive than the others because of that. At some point, you need to start
thinking strategically or you’ll get left behind.</p>
<p>In a workplace, I’d recommend playing Dominion in a tournament format since the
maximum number of players is four.</p>
<h2 id="tabletop-rpgs">Tabletop RPGs</h2>
<p>Tabletop RPGs are also a compelling category for business recreation. The most
popular Tabletop RPG is Dungeons & Dragons, but that’s a massive game which can
sprawl over multiple four hour long sessions for years. I think D&D can be great
for building teams in a business setting, but you need commitment.</p>
<p>D&D is not the only Tabletop RPG, though! I recommend watching the
<a href="https://youtu.be/YqD_oLkrq_8">Tablepop series</a> to get an idea of what shorter
form single session Tabletop RPGs look like.</p>
<p>Here are some of my recommendations beyond D&D. Unfortunately, I don’t have too
many recommendations for Tabletop RPGs that play out over a single session. I’d
like to hear yours!</p>
<ul>
<li><a href="https://www.docdroid.net/KJzmn5k/honey-heist-by-grant-howitt.pdf">Honey Heist</a>:
You’re a bear pulling off a heist to steal honey from Honeycon. This is a
fast, fun, one-shot tabletop RPG.</li>
<li><a href="https://www.evilhat.com/home/monster-of-the-week/">Monster of the Week</a>:
You’re a monster hunter and every week there’s a new threat to fight.
Thematically similar to Buffy the Vampire Slayer and other Monster of the Week
TV shows. Plays out over multiple sessions.</li>
<li><a href="https://en.wikipedia.org/wiki/Vampire:_The_Masquerade">Vampire: The Masquerade</a>:
Players are vampires struggling against their bestial natures. Plays out over
multiple sessions.</li>
</ul>
<p>I also think single session Tabletop RPGs could be a fun addition to a tech
conference. You get some well known speakers on stage and run a session. If
you’re a conference organizer interested in exploring this idea more, let me
know.</p>
<h2 id="conclusion">Conclusion</h2>
<p>If I’m right and boardgames become a mainstream business recreation activity,
then being able to run a boardgame night and introduce new players could be a
valuable skill in a work environment.</p>
<p>So I’m investing into building this skill. I’ll keep you updated on whether or
not it works out for me.</p>My mom is an accountant and a lot of people in her office enjoy playing golf. However, my mom doesn’t play golf so that means to join recreational business outings that involve golf she’s either aimlessly wandering the playing field or not participating at all.My Adventures Writing a Cross-Platform Virtualized List2019-07-09T19:00:00+00:002019-07-09T19:00:00+00:00https://calebmer.com/2019/07/09/my-adventures-writing-a-virtualized-list<p>I wrote a virtualized list! It was quite the adventure.</p>
<p>I was working on a cross-platform React Native app that also rus on the web with
<a href="https://github.com/necolas/react-native-web">React Native Web</a>. None of the
existing virtualized lists were suitable for the product I wanted to build. Not
<a href="http://facebook.github.io/react-native/docs/0.59/flatlist"><code class="language-plaintext highlighter-rouge">FlatList</code></a>, not
<a href="https://react-window.now.sh"><code class="language-plaintext highlighter-rouge">react-window</code></a>.</p>
<p>So, as one does, I wrote my own virtualized list. Forking React Native in the
process. You can see the final code in a
<a href="https://gist.github.com/calebmer/2a3bf40baa929115e7f985f876effb6f">public gist</a>.</p>
<p>I’m going to describe my entire adventure in this post. Through my experience I
hope to inspire you to take control of your code. If writing a virtualized list,
or anything else, would make your user’s life better, you should do it! You need
not be bound to existing libraries. You have the power to fork and modify
dependencies as you see fit. Fear not the unfamiliar, if someone out there wrote
a virtualized list then there’s no reason you can’t!</p>
<p><img class="img-center" src="https://media.giphy.com/media/12vJgj7zMN3jPy/giphy.gif" alt="You can do it GIF" /></p>
<p>This is a story divided into four parts.</p>
<ul>
<li><a href="#part-1-the-product">Part 1: The Product</a></li>
<li><a href="#part-2-when-i-realized-existing-virtualized-lists-wouldnt-work">Part 2: When I realized existing virtualized lists wouldn’t work…</a></li>
<li><a href="#part-3-how-it-works">Part 3: How it works</a></li>
<li><a href="#part-4-forking-react-native">Part 4: Forking React Native</a></li>
</ul>
<h2 id="part-1-the-product">Part 1: The Product</h2>
<p>I was building a React Native Web/iOS/Android app which was, basically, a forum.
A forum has posts and then people could leave comments on that post.</p>
<p>If you were reading the post for the first time, you’d want to read the first
comments and scroll <em>down</em>. If you were catching up on the discussion after
replying, you’d want to read the latest comments and scroll <em>up</em>.</p>
<p>So I needed a virtualized list that would support scrolling from either
direction. I came up with, what I believe, is a new UI pattern: the Skim List! A
sister of the Infinite List.</p>
<p>In a Skim List we pre-allocate space for all the items in the list. When the
user scrolls to a position in the list, we load the items at that position. So
if I scroll 50% through the list, I’ll load items halfway through the list. If I
scroll to the end of the list, I’ll load items at the end of the list.</p>
<p>Here’s the Skim List in action on web. It works the same way on mobile.</p>
<p>These GIFs are slowed down and I added network throttling when recording so you
can really see the progressive loading behavior. It’s really fast and slick when
you get your hands on it.</p>
<p><strong>Scrolling from the top to the bottom</strong></p>
<p><img class="img-center-shrink" src="/assets/images/my-adventures-writing-a-virtualized-list/scroll-down.gif" alt="The Skim List scrolling from the top to the bottom" /></p>
<p><strong>Scrolling from the bottom to the top</strong></p>
<p><img class="img-center-shrink" src="/assets/images/my-adventures-writing-a-virtualized-list/scroll-up.gif" alt="The Skim List scrolling from the bottom to the top" /></p>
<p>As you might be able to imagine, this list also lets you scroll to a random
place in the list and move around.</p>
<h2 id="part-2-when-i-realized-existing-virtualized-lists-wouldnt-work">Part 2: When I realized existing virtualized lists wouldn’t work…</h2>
<p>I first tried using React Native’s
<a href="http://facebook.github.io/react-native/docs/0.59/flatlist"><code class="language-plaintext highlighter-rouge">FlatList</code></a>.</p>
<p>That was working fine, I was able to implement a list where you were able to
scroll down and the comments below you loaded. That’s what <code class="language-plaintext highlighter-rouge">FlatList</code> is
optimized for. However, I also needed the ability to jump to the end and load
comments while scrolling <em>up</em>! <code class="language-plaintext highlighter-rouge">FlatList</code> just wasn’t built for this.</p>
<p>Next I explored <a href="http://react-window.now.sh"><code class="language-plaintext highlighter-rouge">react-window</code></a>. At first glance,
the library obviously wouldn’t work. You need to know the heights of all your
items ahead of time for <code class="language-plaintext highlighter-rouge">react-window</code>. Since I was working with comments on a
post, I had know way of knowing the item heights!</p>
<p>There’s a PR open to
<a href="https://github.com/bvaughn/react-window/issues/6">add a dynamically sized virtualized list for <code class="language-plaintext highlighter-rouge">react-window</code></a>,
but it hadn’t been merged yet.</p>
<p>I needed to incrementally load items in the list when they scrolled into view
and while the items were loading I needed shimmer placeholders. I couldn’t do
this with <code class="language-plaintext highlighter-rouge">FlatList</code> but I could with the unmerged <code class="language-plaintext highlighter-rouge">react-window</code> PR! However, I
needed a solution that would also work on React Native iOS and Android.
<code class="language-plaintext highlighter-rouge">react-window</code> is web only.</p>
<p>Well, that meant I needed to write my own virtualized list.</p>
<h2 id="part-3-how-it-works">Part 3: How it works</h2>
<p>The way my virtualized list works is it takes the total number of items (in this
case comments) on a post and it takes an array of all the comments. I represent
the array as a <em>sparse array</em>. That means any positions in the array without a
loaded comment will be <code class="language-plaintext highlighter-rouge">undefined</code>.</p>
<div class="language-ts highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">type</span> <span class="nx">Props</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">commentCount</span><span class="p">:</span> <span class="kr">number</span><span class="p">;</span>
<span class="nl">comments</span><span class="p">:</span> <span class="nx">ReadonlyArray</span><span class="o"><</span><span class="nx">CommentID</span> <span class="o">|</span> <span class="kc">undefined</span><span class="o">></span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div></div>
<p>For all the comments which are not loaded I render a placeholder component
called <code class="language-plaintext highlighter-rouge"><CommentShimmer></code>. A comment shimmer renders grey boxes which are meant
to look like a conversation. Different comment shimmers have different heights.
I measure the total height of the scroll view with code that roughly looks like
<code class="language-plaintext highlighter-rouge">commentShimmerHeight * commentCount</code>.</p>
<p>I use a pretty standard virtualized list technique. The same one <code class="language-plaintext highlighter-rouge">react-window</code>
uses: absolute positioning. I add a
<a href="https://gist.github.com/calebmer/2a3bf40baa929115e7f985f876effb6f#file-postvirtualizedcomments-tsx-L239">scroll event listener</a>
which calculates the onscreen comments. Then I use
<a href="https://gist.github.com/calebmer/2a3bf40baa929115e7f985f876effb6f#file-postvirtualizedcomments-tsx-L772-L773">absolute positioning</a>
to make sure the comments are rendered at the right position in the virtualized
list.</p>
<p>So whenever a user scrolls I:</p>
<ul>
<li>Figure out which comments to render.</li>
<li>Render <em>only</em> those comments, unmounting any offscreen comments.</li>
<li>Position the rendered comments in the list with absolute positioning.</li>
</ul>
<p>This only works if I know the size of all comments in the list. I know the
height of unloaded comments since they are just <code class="language-plaintext highlighter-rouge"><CommentShimmer></code>s. However,
when a comment loads it might have a completely different height!</p>
<p>When a comment loads I need to measure it. Since I’m using React Native, I must
measure asynchronously. So when the comment is loaded but not measured I render
the <code class="language-plaintext highlighter-rouge"><CommentShimmer></code> and the <code class="language-plaintext highlighter-rouge"><Comment></code> next to each other. Hiding the
<code class="language-plaintext highlighter-rouge"><Comment></code> with <code class="language-plaintext highlighter-rouge">opacity: 0</code>. Once we’ve measured the <code class="language-plaintext highlighter-rouge"><Comment></code> we can get
rid of the <code class="language-plaintext highlighter-rouge"><CommentShimmer></code> and update the height of the list.</p>
<p>So there are three states any comment could be in:</p>
<div class="language-tsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// State 1: Unloaded Comment</span>
<span class="p"><></span>
<span class="p"><</span><span class="nc">CommentShimmer</span> <span class="p">/></span>
<span class="si">{</span><span class="kc">null</span><span class="si">}</span>
<span class="p"></></span>
<span class="c1">// State 2: Loaded but Unmeasured Comment</span>
<span class="p"><></span>
<span class="p"><</span><span class="nc">CommentShimmer</span> <span class="p">/></span>
<span class="p"><</span><span class="nc">View</span> <span class="na">style</span><span class="p">=</span><span class="si">{</span><span class="nx">styles</span><span class="p">.</span><span class="nx">hidden</span><span class="si">}</span> <span class="na">onLayout</span><span class="p">=</span><span class="si">{</span><span class="nx">handleCommentLayout</span><span class="si">}</span><span class="p">></span>
<span class="p"><</span><span class="nc">Comment</span> <span class="p">/></span>
<span class="p"></</span><span class="nc">View</span><span class="p">></span>
<span class="p"></></span>
<span class="c1">// State 3: Loaded and Measured Comment</span>
<span class="p"><></span>
<span class="si">{</span><span class="kc">null</span><span class="si">}</span>
<span class="p"><</span><span class="nc">View</span> <span class="na">style</span><span class="p">=</span><span class="si">{</span><span class="kc">null</span><span class="si">}</span><span class="p">></span>
<span class="p"><</span><span class="nc">Comment</span> <span class="p">/></span>
<span class="p"></</span><span class="nc">View</span><span class="p">></span>
<span class="p"></></span>
</code></pre></div></div>
<p>You can see this
<a href="https://gist.github.com/calebmer/2a3bf40baa929115e7f985f876effb6f#file-postvirtualizedcomments-tsx-L767-L800">in the <code class="language-plaintext highlighter-rouge">renderItem()</code> function</a>.</p>
<h2 id="part-4-forking-react-native">Part 4: Forking React Native</h2>
<p>Ok, at this point the list was working and it was working pretty well. However,
there were a couple bugs I just couldn’t fix. I didn’t just want a <em>good</em>
experience, I wanted a <em>flawless</em> experience. This led me to fork React Native
so I could add a feature to <code class="language-plaintext highlighter-rouge"><ScrollView></code>.</p>
<p>First, let me describe the bug.</p>
<p>When the content of a scroll view resizes, the platform (Web or iOS in this
case) needs to determine where the new scroll position should be. Usually, the
scroll position is measured as the number of pixels that have been scrolled from
the top of the scroll view. So when content resizes, that number is usually kept
constant. See the below image for an example.</p>
<p><img src="/assets/images/my-adventures-writing-a-virtualized-list/pin-window-to-top.png" alt="Behavior of a resized scroll view when we pin the scrolled window to top" /></p>
<p>We change the size of the scroll content, but the scroll window (the red box)
stays the same distance from the top of the scroll view.</p>
<p>This works well in most cases, but it doesn’t work well when the user is
scrolling from bottom to top. That’s because when we load a chunk of comments,
the virtualized list size changes. We add content “above” what the user was
reading which either pushes or pulls the content the user was reading out of the
viewport.</p>
<p>Instead what we want is to pin the scroll window to the <em>bottom</em> of the scroll
view. So when we add new content the distance of the scroll window to the bottom
of the scroll view stays constant. See the below image for an illustration of
the difference.</p>
<p><img src="/assets/images/my-adventures-writing-a-virtualized-list/pin-window-to-bottom.png" alt="Behavior of a resized scroll view when we pin the scrolled window to bottom" /></p>
<p>So I forked React Native and added the <code class="language-plaintext highlighter-rouge">pinWindowTo</code> prop. When set to
<code class="language-plaintext highlighter-rouge">pinWindowTo="top"</code> we use the default behavior. When set to
<code class="language-plaintext highlighter-rouge">pinWindowTo="bottom"</code> it uses the behavior depicted in the previous image.</p>
<p>This is the important part
<a href="https://github.com/calebmer/react-native/commit/8874509405acda979d61504c53cfad4545cae458">of the commit</a>
in the Objective-C code for the <code class="language-plaintext highlighter-rouge">ScrollView</code> component on iOS.</p>
<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code> // offset falls outside of bounds, scroll back to end of list
newOffset.y = MAX(0, newContentSize.height - viewportSize.height);
}
}
+ if (![self.pinWindowTo isEqualToString:@"bottom"]) {
<span class="gi">+ CGFloat oldOffsetBottom = oldContentSize.height - (oldOffset.y + viewportSize.height);
+ newOffset.y = newContentSize.height - viewportSize.height - oldOffsetBottom;
+ }
</span>
BOOL fitsinViewportX = oldContentSize.width <= viewportSize.width && newContentSize.width <= viewportSize.width;
if (newContentSize.width < oldContentSize.width && !fitsinViewportX) {
CGFloat offsetHeight = oldOffset.x + viewportSize.width;```
</code></pre></div></div>
<p>I don’t currently have an Android implementation which is why I haven’t
contributed this back to React Native. In the meantime, this works great for me!</p>
<p>I also implemented this feature on my
<a href="https://github.com/calebmer/react-native-web/commits/master">React Native Web fork</a>.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">_pinWindowToBottom</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">element</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">getScrollableNode</span><span class="p">();</span>
<span class="kd">const</span> <span class="nx">lastScrollTop</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">_lastScrollTop</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">lastScrollHeight</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">_lastScrollHeight</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="nx">_lastScrollHeight</span> <span class="o">=</span> <span class="nx">element</span><span class="p">.</span><span class="nx">scrollHeight</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">lastClientHeight</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">_lastClientHeight</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="nx">_lastClientHeight</span> <span class="o">=</span> <span class="nx">element</span><span class="p">.</span><span class="nx">clientHeight</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">lastScrollBottom</span> <span class="o">=</span> <span class="nx">lastScrollHeight</span> <span class="o">-</span> <span class="p">(</span><span class="nx">lastScrollTop</span> <span class="o">+</span> <span class="nx">lastClientHeight</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">nextScrollTop</span> <span class="o">=</span> <span class="nx">element</span><span class="p">.</span><span class="nx">scrollHeight</span> <span class="o">-</span> <span class="nx">element</span><span class="p">.</span><span class="nx">clientHeight</span> <span class="o">-</span> <span class="nx">lastScrollBottom</span><span class="p">;</span>
<span class="nx">element</span><span class="p">.</span><span class="nx">scrollTop</span> <span class="o">=</span> <span class="nx">nextScrollTop</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="nx">_lastScrollTop</span> <span class="o">=</span> <span class="nx">nextScrollTop</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Other changes I’ve made in my
<a href="https://github.com/calebmer/react-native/commits/0.59-stable">React Native fork</a>:</p>
<ul>
<li>Fixed <a href="https://github.com/facebook/react/issues/15732">React bug</a> until React
and React Native publish a new version.</li>
<li>Send
<a href="https://developer.apple.com/documentation/uikit/uiscrollview/2902259-adjustedcontentinset?language=objc">iOS <code class="language-plaintext highlighter-rouge">adjustedContentInset</code></a>
in scroll events since it’s important for accurate measurements involving
“unsafe areas” on iPhone X.</li>
</ul>
<p>Other changes I‘ve made in my
<a href="https://github.com/calebmer/react-native-web/commits/master">React Native Web</a>
fork:</p>
<ul>
<li>Fire <code class="language-plaintext highlighter-rouge">onLayout</code> in a microtask instead of <code class="language-plaintext highlighter-rouge">setTimeout()</code> so it fires before
the next browser paint. This is very important for my virtualized list double
rendering strategy!</li>
<li>Remove unsafe life-cycle methods like <code class="language-plaintext highlighter-rouge">componentWillReceiveProps</code> so that I
can enable React Concurrent mode in my app.</li>
</ul>
<h3 id="in-defense-of-forking">In defense of forking</h3>
<p>Forking your dependencies is frequently maligned, and for good reason. Without
adequate upkeep your forks will fall behind the latest version of your
dependencies. You’ll miss out on critical bug fixes and security patches!</p>
<p>When I fork I’m very careful to make sure there’s a clear upgrade path in the
future.</p>
<ul>
<li>I only make small changes. The change should only touch a few files and should
be very well documented.</li>
<li>I only make changes which I’d reasonably expect to get merged upstream some
day. That way there’s a path to getting off the fork.</li>
<li>I’ll only make changes I wouldn’t expect to get merged on projects that aren’t
actively maintained.</li>
</ul>
<p>Once I’m comfortable that the change won’t make upgrading too hard in the
future, I fork. Then I have criteria for proposing my forked changes upstream.</p>
<ul>
<li>Is the change tested?</li>
<li>Is the change documented?</li>
<li>Can I show the change working in a production app?</li>
<li>Can I justify the change to contributors?</li>
</ul>
<p>This is a lot of work and slows down shipping. To me, it‘s more valuable to live
on a fork for a few months and fix bugs for users <em>immediately</em> than to make
users wait a few months for a proper open source release with the change.</p>
<p>The best part of open source is that it’s, well, open. You have the power to
modify your dependencies. It’s a dangerous power, but if you use it wisely you
can ship brilliant user experiences no one else is capable of.</p>
<h2 id="conclusion">Conclusion</h2>
<p>As developers, we have so many tools to ship brilliant user experiences. Don’t
be afraid to think out of the box when you encounter a particularly sticky
problem. For me, writing my own virtualized list was the best way to build the
experience I wanted.</p>
<p>Also don’t be afraid of forking your dependencies. Yes it’s dangerous, yes it
will make your life harder if you’re not careful, but it’s also an incredibly
powerful tool. Recognize the risks and use it where appropriate.</p>
<p>I put the code for my
<a href="https://gist.github.com/calebmer/2a3bf40baa929115e7f985f876effb6f">virtualized list in a gist</a>.
I don’t currently plan to turn it into a reusable open source component. That
wasn’t my goal. Shipping a unique experience for my users was my goal.</p>I wrote a virtualized list! It was quite the adventure.Writing Good Compiler Error Messages2019-07-01T19:00:00+00:002019-07-01T19:00:00+00:00https://calebmer.com/2019/07/01/writing-good-compiler-error-messages<p>It’s never fun when you see an error message while trying to ship some code. It
means you’re one step further from working software! It doesn’t help that a lot
of error messages are completely inscrutable.</p>
<p><img src="/assets/images/writing-good-compiler-error-messages/typescript-error.png" alt="A scary TypeScript error" /></p>
<p>After spending a lot of time
<a href="https://medium.com/flow-type/better-flow-error-messages-for-the-javascript-ecosystem-73b6da948ae2">redesigning Flow’s error messages</a>
I’ve developed a personal style guide for writing helpful compiler error
messages. Before jumping into the style guide, I’ll share my philosophy around
error messages.</p>
<h2 id="philosophy">Philosophy</h2>
<p>Roughly, you can sort a developer’s reaction to an error message into two
categories:</p>
<ul>
<li><strong>80%</strong> of the time, the developer will immediately know what the fix is. The
error will be related to code they’re actively working on. The developer may
even know how to fix it <em>just</em> by seeing where the red squiggly in their IDE
is. No need to read the error message.</li>
<li><strong>20%</strong> of the time, the developer won’t immediately know why the compiler
says their code is incorrect. The compiler might have a complex rule that’s
non-intuitive but important to the safety of the program. In this case, an
error message should provide enough information to enable the developer to
take a deep dive into their program.</li>
</ul>
<p>In the <strong>80%</strong> case a developer is either quickly iterating through their code
and a long error message would be a <em>disservice</em> to their productivity. They
only need a glance at the message to know what to fix.</p>
<p>In the <strong>20%</strong> case it will be really difficult to pack enough context into a
single error message to explain the problem. The error message’s fix might be
somewhere far away from the reported location since the compiler can’t determine
a human’s intent. The error might be caused by a really nuanced rule that can’t
easily be explained in an error message format.</p>
<p>A great example of a rough error messages in the 20% case is anything reported
by the Rust borrow checker or the Elm infinite type error message:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>I am inferring a weird self-referential type for x:
11| f x = x x
^
Here is my best effort at writing down the type. You will see ∞ for parts of
the type that repeat something already printed out infinitely.
∞ -> a
Staring at this type is usually not so helpful, so I recommend reading the
hints at <https://elm-lang.org/0.19.0/infinite-type> to get unstuck!
</code></pre></div></div>
<p>Elm even acknowledges that “staring at this…is usually not helpful”.</p>
<h2 id="context">Context</h2>
<p>It’s also important to understand the context of <em>where</em> the developer will see
an error message. For a programming language, ideally you have good editor
integration so the user is going to see your error message in an editor like
VSCode.</p>
<p>The IDE context favors short error messages that are one to two sentences long.
When you hover over an error the popup doesn’t give you much space!</p>
<p><img src="/assets/images/writing-good-compiler-error-messages/error-popup.png" alt="VSCode Error Popup" /></p>
<p>Many IDEs, like VSCode, also have some kind of panel where they show you all the
error messages at once. The design of this space is also best suited to short
one to two sentence long error messages.</p>
<p><img src="/assets/images/writing-good-compiler-error-messages/problems-pane.png" alt="VSCode Problems Pane" /></p>
<p>Thinking about the context where an error message is displayed influences my
style guide <em>a lot</em>. Usually, most compiler developers are designing their error
messages in the command line (CLI). Which is why you get colored multiline
messages that print out code.</p>
<p>I design error messages <em>IDE first</em>, not <em>CLI first</em>. Like the way you’d design
a website to be mobile first. The IDE is much more constrained than the CLI. If
you design a good IDE message you can easily adapt it to the CLI. It’s much
harder to adapt a message designed for the CLI to an IDE since you didn’t think
of an IDE’s constraints when designing your message.</p>
<h2 id="style-guide">Style Guide</h2>
<p>Now that you know my philosophy and the context I design for, let’s get into my
error message style guide!</p>
<ul>
<li>
<p>Keep error messages short. Preferably a single, clear, sentence. This format
works best in an IDE context.</p>
</li>
<li>
<p>The error location is so important since that’s where the red squiggly goes in
an IDE. Pick the smallest possible location at the operation which triggered
the error. If the user is writing a new operation the error will point to
where they are working. If the user is refactoring the error will point to all
the operations which need to change.</p>
</li>
<li>
<p>Don’t print out information a developer could easily find in their code.
Instead print a reference to that information which is linked in an IDE.
TypeScript likes to print out huge types in error messages, this makes it hard
to read the message.</p>
</li>
<li>
<p>Use correct English grammar. This is probably obvious, but it can be hard to
make a program which produces correct English grammar. It’s easy to stitch
together arbitrary sentence fragments. Put some rules on the grammatical
structure of your sentence fragments. It’s also worth going the extra mile to
add grammatical decoration like articles: “a <code class="language-plaintext highlighter-rouge">String</code> is not an <code class="language-plaintext highlighter-rouge">Int</code>” vs
“<code class="language-plaintext highlighter-rouge">String</code> is not <code class="language-plaintext highlighter-rouge">Int</code>”. Just be careful when using developer-defined names.</p>
</li>
<li>
<p>Write messages in first person plural. That is, use “we”. For example “we see
an error”. This personifies the compiler as a <em>team</em> of people looking for
bugs in the developer’s code. By personifying our type checker error messages
feel like a dialogue. Elm’s error messages are famous for using first person:
“I see an error”. First person feels a bit uncomfortable to me. A compiler is
certainly not a single person, nor is it built by a single person. I prefer
“we” as a compromise.</p>
</li>
<li>
<p>Use present tense instead of past tense. Instead of “we found” say “we see”.
When an error is displayed to a user, the code is currently in a bad state.
From the compiler author’s perspective, the compiler runs at discrete points
in time and “finds” errors at those points. From the developer’s perspective
an error in their IDE reflects the current state of the program, not a
discrete compiler run. Prefer the developer’s IDE context, use present tense.</p>
</li>
<li>
<p>Reduce the grade reading level as much as possible. I use the
<a href="http://www.hemingwayapp.com/">Hemingway Editor</a> for this. I’ll paste in an
error message and the editor tells me the grade reading level (for example,
6th grade). I then try to reduce the grade level as much as possible without
sacrificing clarity.</p>
</li>
<li>
<p>Use language the developer will understand, not compiler-speak. Words like
“identifier”, “token”, and “expression” are compiler-speak. Prefer the names a
developer would use in conversation with their peers. Also consider the
context of your error message. If you expect a variable name, say that instead
of saying “expected identifier”.</p>
</li>
<li>
<p>Use Markdown for formatting error messages. If you want to include a code
snippet put it between backticks (`) which are used in Markdown to indicate
inline code. The code editor should format your message appropriately.</p>
</li>
<li>
<p>If you use quotes, make sure they are the proper curly quote characters. For
example: “phrase” instead of "phrase". See the difference? It’s subtle. I’ve
gotten into the habit of typing these characters after taking a typography
course. <a href="https://graphemica.com/%E2%80%9C">U+201C (“)</a>,
<a href="https://graphemica.com/%E2%80%9D">U+201D (”)</a>,
<a href="https://graphemica.com/%E2%80%98">U+2018 (‘)</a>, and
<a href="https://graphemica.com/%E2%80%99">U+2019 (’)</a>.</p>
</li>
</ul>
<p>Following this guide will produce error messages with a consistent tone, voice,
and style.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Short, simple, and clear error messages are <em>much</em> better than long and detailed
error messages. They fit the developer’s IDE context better and they don’t
distract the developer when they are trying to debug a particularly unintuitive
error.</p>
<p>Good error messages will make developers more efficient. They’ll spend less time
staring at error message text and more time building products their users will
love.</p>
<h2 id="ide-wishlist">IDE Wishlist</h2>
<p>There are two features I want from the IDEs rendering my error messages written
with my style guide. Specifically, I want these features in the
<a href="https://microsoft.github.io/language-server-protocol/specification">Language Server Protocol (LSP) Specification</a>
which is supported by VSCode, Atom, and other code editors.</p>
<ol>
<li>I want my error messages to be rendered with markdown. Currently, at least in
VSCode, error messages are rendered with a monospace font. My messages are
designed to be proper English sentences with inline code snippets, not code.</li>
<li>I want to link to code in error messages. Often, I’ll have an error that
reads “<code class="language-plaintext highlighter-rouge">Foo</code> is not <code class="language-plaintext highlighter-rouge">Bar</code>”. I want “<code class="language-plaintext highlighter-rouge">Foo</code>” to be a hyperlink to where the
compiler found the <code class="language-plaintext highlighter-rouge">Foo</code> type! Same for <code class="language-plaintext highlighter-rouge">Bar</code>. I can hack around this with
the
<a href="https://microsoft.github.io/language-server-protocol/specification#diagnostic">LSP’s <code class="language-plaintext highlighter-rouge">relatedInformation</code> field</a>
but it’s not the same as being able to write a hyperlink inline.</li>
</ol>
<h2 id="further-reading">Further Reading</h2>
<ul>
<li>My announcement for
<a href="https://medium.com/flow-type/better-flow-error-messages-for-the-javascript-ecosystem-73b6da948ae2">Flow’s error message redesign</a>.</li>
<li>Elm’s
<a href="https://elm-lang.org/blog/compiler-errors-for-humans">compiler errors for humans</a>
post.</li>
<li>Reason has also given a
<a href="https://reasonml.github.io/blog/2017/08/25/way-nicer-error-messages.html">lot of thought</a>
towards their error messages.</li>
</ul>It’s never fun when you see an error message while trying to ship some code. It means you’re one step further from working software! It doesn’t help that a lot of error messages are completely inscrutable.Taking Control of Compilation2019-06-18T03:00:00+00:002019-06-18T03:00:00+00:00https://calebmer.com/2019/06/18/taking-control-of-compilation<p>The more work we can do at compile time, the better! That’s less work our code
has to do at runtime, but it’s not easy to move work to compile time. In this
post I propose a new way of executing code, focused on JavaScript applications,
that makes <em>moving work to compile time as simple as moving a variable to global
scope</em>.</p>
<p>Let’s first think about our code’s compilation and execution in two phases,
<strong>buildtime</strong> and <strong>runtime</strong> respectively.</p>
<p>Buildtime code executes on the developer’s machine before they deploy their
application. Runtime code executes on a user’s machine when they run the
application.</p>
<p>Buildtime tasks include Webpack bundling many JavaScript files into one big file
or Babel compiling ES6 features down to ES5. <a href="https://svelte.dev/">Svelte</a> does
a lot of work at buildtime to create a fast application at runtime.</p>
<p>When we developers write JavaScript, or other product languages like Swift and
Kotlin, we expect that all our code will be executed at runtime. If we want to
execute something at buildtime we write a script or a compiler plugin.</p>
<p>There are a lot of tasks in product development which would make sense to run at
buildtime, but we often run these tasks at runtime instead since it’s
convenient. For example:</p>
<ul>
<li>Parsing <a href="https://github.com/apollographql/graphql-tag">Apollo GraphQL</a>
queries.</li>
<li>Parsing <a href="https://www.styled-components.com/">Styled Components</a> CSS.</li>
<li>Choosing a translation based on the locale.</li>
</ul>
<p>There are also a lot of tasks which we do run at buildtime but require lots of
configuration <em>outside</em> of our source code.</p>
<ul>
<li>Bundle splitting.</li>
<li>Optimizing image formats like SVG.</li>
<li>Extracting CSS to a static file.</li>
<li>Generating HTML files.</li>
</ul>
<p>We have many opportunities to optimize our applications at buildtime. However,
we miss a lot of those opportunities because configuring Webpack or Babel or
<em>insert buildtime tool here</em> is hard.</p>
<p>We also have very few ways to innovate when it to buildtime work. There’s
<a href="https://svelte.dev">Svelte</a> which had to build its own compiler to innovate at
buildtime. There’s
<a href="https://github.com/kentcdodds/babel-plugin-macros"><code class="language-plaintext highlighter-rouge">babel-plugin-macros</code></a> which
requires macro authors to write a Babel AST transform where it’s easy to miss
corner cases.</p>
<p>Today, I will propose a vision for a new way to execute programs that makes
writing buildtime code <em>free</em>. No Webpack plugins. No Babel transforms. No
custom compilers.</p>
<p>By making buildtime code dead simple to write, we can empower every developer to
make the next big compiler optimization for our products.</p>
<h2 id="vision-modules-execute-at-buildtime-not-runtime">Vision: Modules execute at Buildtime, not Runtime</h2>
<p>Today a JavaScript module that looks like this:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">fibonacci</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./utils/fibonacci</span><span class="dl">'</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">fibonacci</span><span class="p">(</span><span class="mi">12</span><span class="p">));</span> <span class="c1">// 144</span>
</code></pre></div></div>
<p>Will print “144” to the console of every end user of the application. This
module executes at <em>runtime</em>.</p>
<p>Instead, let’s execute it at buildtime and add a hook for executing code at
runtime.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">fibonacci</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./utils/fibonacci</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">x</span> <span class="o">=</span> <span class="nx">fibonacci</span><span class="p">(</span><span class="mi">12</span><span class="p">);</span>
<span class="nx">Build</span><span class="p">.</span><span class="nx">createBundle</span><span class="p">(</span><span class="dl">'</span><span class="s1">my-bundle.js</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">x</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>We now execute this module at buildtime. We compute <code class="language-plaintext highlighter-rouge">fibonacci(12)</code> once on the
developer’s machine instead of thousands of times on the machines of their
users.</p>
<p>Then, we call <code class="language-plaintext highlighter-rouge">Build.createBundle()</code> which creates a new JS bundle named
<code class="language-plaintext highlighter-rouge">my-bundle.js</code>. Just like adding a new entry to your Webpack config. After
running this code at buildtime you’ll get a bundle that looks like this.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>var x = 144;
console.log(x);
</code></pre></div></div>
<p>We don’t have to include our <code class="language-plaintext highlighter-rouge">fibonacci()</code> function in the bundle and we can
directly inline the constant result of calling <code class="language-plaintext highlighter-rouge">fibonacci(12)</code>.</p>
<p>Let’s consider a more advanced example, translations.</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">App</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./App</span><span class="dl">'</span><span class="p">;</span>
<span class="k">import</span> <span class="nx">TranslationContext</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./TranslationContext</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">localeNames</span> <span class="o">=</span> <span class="nx">Build</span><span class="p">.</span><span class="nx">importDirectory</span><span class="p">(</span><span class="dl">'</span><span class="s1">./translations/locales</span><span class="dl">'</span><span class="p">);</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">const</span> <span class="nx">localeName</span> <span class="k">of</span> <span class="nx">localeNames</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">locale</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span>
<span class="nx">Build</span><span class="p">.</span><span class="nx">importFile</span><span class="p">(</span><span class="s2">`./translations/locales/</span><span class="p">${</span><span class="nx">localeName</span><span class="p">}</span><span class="s2">.json`</span><span class="p">)</span>
<span class="p">);</span>
<span class="nx">Build</span><span class="p">.</span><span class="nx">createBundle</span><span class="p">(</span><span class="s2">`app.</span><span class="p">${</span><span class="nx">localeName</span><span class="p">}</span><span class="s2">.js`</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">React</span><span class="p">.</span><span class="nx">render</span><span class="p">(</span>
<span class="p"><</span><span class="nc">TranslationContext</span><span class="p">.</span><span class="nc">Provider</span> <span class="na">value</span><span class="p">=</span><span class="si">{</span><span class="nx">locale</span><span class="si">}</span><span class="p">></span>
<span class="p"><</span><span class="nc">App</span> <span class="p">/></span>
<span class="p"></</span><span class="nc">TranslationContext</span><span class="p">.</span><span class="nc">Provider</span><span class="p">></span>
<span class="p">);</span>
<span class="p">});</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Let’s say you have a directory <code class="language-plaintext highlighter-rouge">./translations/locales</code> with three files that
include translations for your app: <code class="language-plaintext highlighter-rouge">en-US.json</code>, <code class="language-plaintext highlighter-rouge">es-MX.json</code>, and <code class="language-plaintext highlighter-rouge">zh-CN.json</code>.</p>
<p>In this example, we declare that our app depends on all the files in our
translations directory using <code class="language-plaintext highlighter-rouge">Build.importDirectory()</code> and <code class="language-plaintext highlighter-rouge">Build.importFile()</code>.
A good buildtime framework can also then watch these JSON files and rebuild the
bundles when they change.</p>
<p>We then create a new bundle for every locale where we render a React app! Since
the arrow function captures the <code class="language-plaintext highlighter-rouge">locale</code> JSON data, that data will be included
in the final bundle. Along with any other constant data we capture.</p>
<p>If you have a page that doesn’t require any data from the user, you may also
compile it statically at buildtime. Like <a href="https://www.gatsbyjs.org/">Gatsby</a>!</p>
<p>It‘s not just our top level application files that run at buildtime. <em>Every one
of our files would run at buildtime.</em></p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">export</span> <span class="kd">function</span> <span class="nx">TodoItem</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">data</span> <span class="o">=</span> <span class="nx">useQuery</span><span class="p">(</span><span class="nx">query</span><span class="p">);</span>
<span class="k">return</span> <span class="p"><</span><span class="nt">div</span> <span class="na">className</span><span class="p">=</span><span class="si">{</span><span class="nx">styles</span><span class="p">.</span><span class="nx">item</span><span class="si">}</span><span class="p">></span><span class="si">{</span><span class="cm">/* ... */</span><span class="si">}</span><span class="p"></</span><span class="nt">div</span><span class="p">>;</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">query</span> <span class="o">=</span> <span class="nx">graphql</span><span class="s2">`
fragment todo on TodoItem {
id
name
completed
# ...
}
`</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">styles</span> <span class="o">=</span> <span class="nx">css</span><span class="s2">`
.item {
border: solid 2px red;
}
/* ... */
`</span><span class="p">;</span>
</code></pre></div></div>
<p>In this example, we have two variables in our module scope: <code class="language-plaintext highlighter-rouge">query</code> and
<code class="language-plaintext highlighter-rouge">styles</code>. Both of these variables would be computed at buildtime! We’d parse the
corresponding GraphQL and CSS, run any transformations, and output JSON objects
to the resulting bundle.</p>
<p>The only difference is we’d do it at buildtime instead of runtime.</p>
<p>To use a React component defined like this:</p>
<div class="language-jsx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">TodoItem</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">./TodoItem</span><span class="dl">'</span><span class="p">;</span>
<span class="c1">// Nope! We don’t render in the module scope.</span>
<span class="c1">//</span>
<span class="c1">// ❌ React.render(todos.map(todo => <TodoItem todo={todo} />));</span>
<span class="c1">// Instead we render in a bundle we create at buildtime.</span>
<span class="nx">Build</span><span class="p">.</span><span class="nx">createBundle</span><span class="p">(</span><span class="dl">'</span><span class="s1">app.js</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">React</span><span class="p">.</span><span class="nx">render</span><span class="p">(</span><span class="nx">todos</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">todo</span> <span class="o">=></span> <span class="p"><</span><span class="nc">TodoItem</span> <span class="na">todo</span><span class="p">=</span><span class="si">{</span><span class="nx">todo</span><span class="si">}</span> <span class="p">/>));</span>
<span class="p">});</span>
</code></pre></div></div>
<h2 id="interested">Interested?</h2>
<p>Does this model of execution interest you? Let me know! Twitter:
<a href="https://twitter.com/calebmer">@calebmer</a>.</p>
<p>There’s a lot to figure out to make this work in JavaScript. I imagine we could
create an AST transform that adds an environment record of the variables
captured a function. So the following code:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">x</span> <span class="o">=</span> <span class="nx">fibonacci</span><span class="p">(</span><span class="mi">10</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">y</span> <span class="o">=</span> <span class="nx">fibonacci</span><span class="p">(</span><span class="mi">11</span><span class="p">);</span>
<span class="nx">Build</span><span class="p">.</span><span class="nx">createBundle</span><span class="p">(</span><span class="dl">'</span><span class="s1">my-bundle.js</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">x</span> <span class="o">+</span> <span class="nx">y</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>Would be transformed by wrapping all functions in some <code class="language-plaintext highlighter-rouge">addCapturedVariables()</code>
function:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">x</span> <span class="o">=</span> <span class="nx">fibonacci</span><span class="p">(</span><span class="mi">10</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">y</span> <span class="o">=</span> <span class="nx">fibonacci</span><span class="p">(</span><span class="mi">11</span><span class="p">);</span>
<span class="nx">Build</span><span class="p">.</span><span class="nx">createBundle</span><span class="p">(</span>
<span class="dl">'</span><span class="s1">my-bundle.js</span><span class="dl">'</span><span class="p">,</span>
<span class="nx">addCapturedVariables</span><span class="p">({</span><span class="na">x</span><span class="p">:</span> <span class="nx">x</span><span class="p">,</span> <span class="na">y</span><span class="p">:</span> <span class="nx">y</span><span class="p">},</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">x</span> <span class="o">+</span> <span class="nx">y</span><span class="p">);</span>
<span class="p">})</span>
<span class="p">);</span>
</code></pre></div></div>
<p>That way buildtime code could serialize captured variables out to a runtime
bundle.</p>
<p>Making this process fast is another question entirely. There are likely ways to
intelligently determine if buildtime or runtime code was changed. If only
runtime code changed, then you don’t need to re-run your buildtime logic.</p>
<h2 id="further-reading">Further Reading</h2>
<ul>
<li><a href="https://github.com/facebook/prepack">Prepack</a> kinda does something like this.
The big difference is that Prepack tries to optimize code you already have
with a custom JavaScript engine. This proposal would reuse existing JavaScript
engines and add functions for exploiting the power of buildtime evaluation
like <code class="language-plaintext highlighter-rouge">Build.createBundle()</code>.</li>
<li>The idea of compile-time function execution already exists in Lisp macros, in
C++ as <a href="https://en.cppreference.com/w/cpp/language/constexpr"><code class="language-plaintext highlighter-rouge">constexpr</code></a>,
and in Rust as
<a href="https://doc.rust-lang.org/unstable-book/language-features/const-fn.html"><code class="language-plaintext highlighter-rouge">const fn</code></a>.
These systems allow you to execute pure functions with static inputs, but
nothing I’ve seen has the ability to customize your build like
<code class="language-plaintext highlighter-rouge">Build.createBundle()</code> would.</li>
<li><a href="https://anydsl.github.io/">AnyDSL</a> is a cool project that does deep partial
evaluation like Prepack. It comes with a new programming language and an
interesting graph-based CPS intermediate representation called
<a href="https://anydsl.github.io/Thorin.html">Thorin</a>.</li>
</ul>The more work we can do at compile time, the better! That’s less work our code has to do at runtime, but it’s not easy to move work to compile time. In this post I propose a new way of executing code, focused on JavaScript applications, that makes moving work to compile time as simple as moving a variable to global scope.Pull-Based Reactivity Primitive2019-02-06T16:00:00+00:002019-02-06T16:00:00+00:00https://calebmer.com/2019/02/06/pull-based-reactivity-primitive<p>Observables, like those provided by <a href="http://reactivex.io">Reactive Extensions</a>, are a <em>push</em>
based reactivity primitive. <a href="https://reactjs.org">React</a>, on the other hand, uses a <em>pull</em>
based reactivity model. I’ll be using JavaScript and <a href="https://rxjs.dev">RxJS</a> for my
examples.</p>
<p>An Observable <em>pushes</em> events over time. A bit like an event emitter.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">button</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">querySelector</span><span class="p">(</span><span class="dl">'</span><span class="s1">button</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">fromEvent</span><span class="p">(</span><span class="nx">button</span><span class="p">,</span> <span class="dl">'</span><span class="s1">click</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">pipe</span><span class="p">(</span><span class="nx">scan</span><span class="p">(</span><span class="nx">count</span> <span class="o">=></span> <span class="nx">count</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
<span class="p">.</span><span class="nx">subscribe</span><span class="p">(</span><span class="nx">count</span> <span class="o">=></span> <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Clicked </span><span class="p">${</span><span class="nx">count</span><span class="p">}</span><span class="s2"> times`</span><span class="p">));</span>
</code></pre></div></div>
<p>In this example, every time a <code class="language-plaintext highlighter-rouge"><button></code> is clicked we <em>push</em> a new event. Every
event is guaranteed to be seen by our subscriber. It’s as if we implemented the
above code with:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">button</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">querySelector</span><span class="p">(</span><span class="dl">'</span><span class="s1">button</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">let</span> <span class="nx">count</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">'</span><span class="s1">click</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">count</span> <span class="o">=</span> <span class="nx">count</span> <span class="o">+</span> <span class="mi">1</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Clicked </span><span class="p">${</span><span class="nx">count</span><span class="p">}</span><span class="s2"> times`</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>React uses a pull-based reactivity model which is subtly different. I’ll be
using <a href="https://reactjs.org/docs/hooks-intro.html">React Hooks</a> in my examples.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">MyButton</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">let</span> <span class="p">[</span><span class="nx">count</span><span class="p">,</span> <span class="nx">setCount</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="kd">function</span> <span class="nx">handleClick</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">setCount</span><span class="p">(</span><span class="nx">prevCount</span> <span class="o">=></span> <span class="nx">prevCount</span> <span class="o">+</span> <span class="mi">1</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">return</span> <span class="o"><</span><span class="nx">button</span> <span class="nx">onClick</span><span class="o">=</span><span class="p">{</span><span class="nx">handleClick</span><span class="p">}</span><span class="o">></span><span class="p">{</span><span class="nx">count</span><span class="p">}</span><span class="o"><</span><span class="nx">button</span><span class="o">/></span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">setCount()</code> function does not <em>push</em> a new state to the React component.
Instead, it invalidates the current state and lets React update the component
whenever it wants. (The same is true for <code class="language-plaintext highlighter-rouge">setState()</code> in a class component.)</p>
<p>It’s like we wrote:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">button</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">querySelector</span><span class="p">(</span><span class="dl">'</span><span class="s1">button</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">let</span> <span class="nx">count</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">'</span><span class="s1">click</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">count</span> <span class="o">=</span> <span class="nx">count</span> <span class="o">+</span> <span class="mi">1</span><span class="p">;</span>
<span class="nx">invalidate</span><span class="p">();</span>
<span class="p">});</span>
<span class="kd">let</span> <span class="nx">isInvalid</span> <span class="o">=</span> <span class="kc">false</span><span class="p">;</span>
<span class="kd">function</span> <span class="nx">invalidate</span><span class="p">()</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">isInvalid</span> <span class="o">===</span> <span class="kc">false</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">isInvalid</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="nx">scheduler</span><span class="p">.</span><span class="nx">withUserInteractionPriority</span><span class="p">(()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">button</span><span class="p">.</span><span class="nx">innerText</span> <span class="o">=</span> <span class="nx">count</span><span class="p">.</span><span class="nx">toString</span><span class="p">();</span>
<span class="nx">isInvalid</span> <span class="o">=</span> <span class="kc">false</span><span class="p">;</span>
<span class="p">});</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If you’ve written React for a while, you’ve probably run into this:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">handleClick</span><span class="p">()</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">count</span><span class="p">);</span> <span class="c1">// Prints 0</span>
<span class="nx">setCount</span><span class="p">(</span><span class="nx">prevCount</span> <span class="o">=></span> <span class="nx">prevCount</span> <span class="o">+</span> <span class="mi">1</span><span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">count</span><span class="p">);</span> <span class="c1">// Prints 0</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Even though you called <code class="language-plaintext highlighter-rouge">setCount()</code>, the value for <code class="language-plaintext highlighter-rouge">count</code> does not change! The
value for <code class="language-plaintext highlighter-rouge">count</code> will change <em>eventually</em> but not immediately. This is because
when you call <code class="language-plaintext highlighter-rouge">setCount()</code>, React in Concurrent Mode doesn’t immediately update
your component. Instead, it <em>schedules</em> an update to happen at some later time.</p>
<p>If the action is user-generated, like a click, React will schedule an update
with a higher priority. If the action is not user-generated, like you just got a
big JSON blob from your backend, React will schedule an update with a lower
priority.</p>
<p>This means if a user clicks at the same time you get a big JSON blob from your
backend the user’s click will always be prioritized and rendered <em>first</em>.</p>
<p>This just isn’t possible in a push-based reactivity system. In a push-based
reactivity system the producer has control over when an action is processed. If
I push you something, you <em>must</em> process it. In a pull-based reactivity system
the consumer has control over when an action is processed. If I push you
something, I don’t get to tell you when it is processed.</p>
<p>Concurrent React takes full advantage of this control by using a
<a href="https://github.com/facebook/react/blob/c21c41ecfad46de0a718d059374e48d13cf08ced/packages/scheduler/src/Scheduler.js">scheduler</a> to prioritize user actions over every other action.</p>
<!-- prettier-ignore-start -->
<!-- prettier-ignore-end -->
<h2 id="rust-futures">Rust Futures</h2>
<p><a href="https://docs.rs/futures/0.1.25/futures">Rust futures</a> are another example of pull-based reactivity.
Futures in Rust are like promises in JavaScript. However, while a JavaScript
promise is push-based a Rust future is pull-based. Like most things in Rust,
futures aim to be a zero-cost abstraction for async IO.</p>
<p>For a data type in Rust to be a future it must implement this trait:</p>
<div class="language-rs highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">trait</span> <span class="n">Future</span> <span class="p">{</span>
<span class="k">type</span> <span class="n">Item</span><span class="p">;</span>
<span class="k">type</span> <span class="n">Error</span><span class="p">;</span>
<span class="k">fn</span> <span class="nf">poll</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span> <span class="k">self</span><span class="p">)</span> <span class="k">-></span> <span class="n">Poll</span><span class="o"><</span><span class="nn">Self</span><span class="p">::</span><span class="n">Item</span><span class="p">,</span> <span class="nn">Self</span><span class="p">::</span><span class="n">Error</span><span class="o">></span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The type <a href="https://docs.rs/futures/0.1.25/futures/type.Poll.html"><code class="language-plaintext highlighter-rouge">Poll<Item, Error></code></a> type may have one of three states:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Ok(Async::Ready(item))</code> means that a future has successfully resolved.</li>
<li><code class="language-plaintext highlighter-rouge">Ok(Async::NotReady)</code> means that a future is not ready to complete yet.</li>
<li><code class="language-plaintext highlighter-rouge">Err(error)</code> means that a future has completed with the given failure.</li>
</ul>
<p>These are similar to the three states of a JavaScript promise: resolved,
pending, and rejected. Except you create a JavaScript promise with:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">new</span> <span class="nb">Promise</span><span class="p">((</span><span class="nx">resolve</span><span class="p">,</span> <span class="nx">reject</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="p">});</span>
</code></pre></div></div>
<p>Calling <code class="language-plaintext highlighter-rouge">resolve()</code> <em>pushes</em> the resolved value to anyone listening to the
promise. In Rust, however, the consumer must continue to call <code class="language-plaintext highlighter-rouge">poll()</code> while it
returns <code class="language-plaintext highlighter-rouge">Ok(Async::NotReady)</code>. To resolve a Rust future and block the current
thread one might write:</p>
<div class="language-rs highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">loop</span> <span class="p">{</span>
<span class="k">match</span> <span class="n">future</span><span class="nf">.poll</span><span class="p">()</span> <span class="p">{</span>
<span class="nf">Ok</span><span class="p">(</span><span class="nn">Async</span><span class="p">::</span><span class="n">NotReady</span><span class="p">)</span> <span class="k">=></span> <span class="p">{}</span> <span class="c">// Try again...</span>
<span class="nf">Ok</span><span class="p">(</span><span class="nn">Async</span><span class="p">::</span><span class="nf">Ready</span><span class="p">(</span><span class="n">item</span><span class="p">))</span> <span class="k">=></span> <span class="k">return</span> <span class="nf">Ok</span><span class="p">(</span><span class="n">item</span><span class="p">),</span>
<span class="nf">Err</span><span class="p">(</span><span class="n">error</span><span class="p">)</span> <span class="k">=></span> <span class="k">return</span> <span class="nf">Err</span><span class="p">(</span><span class="n">error</span><span class="p">),</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This code will loop until the future is either resolved or rejected.</p>
<p>Rust futures also come with a way to register a task. If you call
<code class="language-plaintext highlighter-rouge">future.poll()</code> and get <code class="language-plaintext highlighter-rouge">Async::NotReady</code> the <code class="language-plaintext highlighter-rouge">poll()</code> function will notify your
task when the value is ready. If futures were written in JavaScript we’d write a
future for fetching some data like this:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">fetchStarted</span> <span class="o">=</span> <span class="kc">false</span><span class="p">;</span>
<span class="kd">let</span> <span class="nx">currentTask</span><span class="p">;</span>
<span class="kd">let</span> <span class="nx">result</span><span class="p">;</span>
<span class="kd">function</span> <span class="nx">poll</span><span class="p">()</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">result</span> <span class="o">!==</span> <span class="kc">undefined</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">result</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">fetchStarted</span> <span class="o">===</span> <span class="kc">false</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">fetchStarted</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="nx">fetch</span><span class="p">(</span><span class="dl">'</span><span class="s1">https://api.example.org/user/42</span><span class="dl">'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">then</span><span class="p">(</span><span class="nx">response</span> <span class="o">=></span> <span class="nx">response</span><span class="p">.</span><span class="nx">json</span><span class="p">())</span>
<span class="p">.</span><span class="nx">then</span><span class="p">(</span><span class="nx">responseData</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">result</span> <span class="o">=</span> <span class="nx">responseData</span><span class="p">;</span>
<span class="c1">// If `poll()` was called in the context of a task, notify that task</span>
<span class="c1">// so it calls `poll()` again.</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">currentTask</span> <span class="o">!==</span> <span class="kc">undefined</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">currentTask</span><span class="p">.</span><span class="nx">notify</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">});</span>
<span class="p">}</span>
<span class="c1">// Record the current task and tell the caller of `poll()` that we</span>
<span class="c1">// aren’t ready.</span>
<span class="nx">currentTask</span> <span class="o">=</span> <span class="nx">Task</span><span class="p">.</span><span class="nx">current</span><span class="p">();</span>
<span class="k">return</span> <span class="nx">Async</span><span class="p">.</span><span class="nx">NotReady</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>When <code class="language-plaintext highlighter-rouge">poll()</code> is called the first time we start our <code class="language-plaintext highlighter-rouge">fetch()</code> request and we
return <code class="language-plaintext highlighter-rouge">Async.NotReady</code>. If <code class="language-plaintext highlighter-rouge">poll()</code> is called again before <code class="language-plaintext highlighter-rouge">fetch()</code> finishes
then we return <code class="language-plaintext highlighter-rouge">Async.NotReady</code> again. Finally, when we get our response back,
we will call <code class="language-plaintext highlighter-rouge">currentTask.notify()</code> which tells our task that we are done.</p>
<p>Now that <code class="language-plaintext highlighter-rouge">currentTask</code> was notified, it can call <code class="language-plaintext highlighter-rouge">poll()</code> again <em>whenever</em> it
wants. The task can use a scheduler to wait to call <code class="language-plaintext highlighter-rouge">poll()</code> until after all
user actions are finished, for instance. This is what makes Rust futures poll
based.</p>
<p>Remember that Rust futures, like promises, are one-shot. They resolve to a
single value instead of multiple values that change over time.</p>
<h2 id="primitive">Primitive</h2>
<p>The common feature between React’s implementation of pull-based reactivity and
Rust future’s implementation of pull-based reactivity is <em>invalidation</em>. Instead
of pushing new values to listeners, these implementations will tell listeners
when there is a new value.</p>
<p>There are two fundamental operations to the pull-based reactivity primitive:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">get()</code> which retrieves the value and possibly does some computation.</li>
<li><code class="language-plaintext highlighter-rouge">invalidate()</code> which tells listeners that the value has changed.</li>
</ul>
<p>You can imagine your data source using this primitive. <code class="language-plaintext highlighter-rouge">get()</code> fetches a user
profile and when the user edits their profile you’d call <code class="language-plaintext highlighter-rouge">invalidate()</code> on that
data source. Now everyone who was using the user profile may call <code class="language-plaintext highlighter-rouge">get()</code> to
re-fetch the new user profile.</p>
<p>In React land, <code class="language-plaintext highlighter-rouge">get()</code> is when React calls your component’s render function. It
is <em>getting</em> the latest UI. You <code class="language-plaintext highlighter-rouge">invalidate()</code> your component by calling a
<code class="language-plaintext highlighter-rouge">setState()</code> function.</p>
<p>In Rust future land, <code class="language-plaintext highlighter-rouge">get()</code> is the <code class="language-plaintext highlighter-rouge">poll()</code> function and <code class="language-plaintext highlighter-rouge">invalidate()</code> is
<code class="language-plaintext highlighter-rouge">task.notify()</code> which tells the task it’s time to call <code class="language-plaintext highlighter-rouge">poll()</code> again.</p>
<h2 id="implementation-deep-end">Implementation: Deep End</h2>
<p>If we go to implement our pull-based reactivity in JavaScript we may end up with
something that looks like this.</p>
<div class="language-ts highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">let</span> <span class="nx">invalid</span> <span class="o">=</span> <span class="nb">Symbol</span><span class="p">(</span><span class="dl">'</span><span class="s1">invalid</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">class</span> <span class="nx">State</span><span class="o"><</span><span class="nx">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">private</span> <span class="k">readonly</span> <span class="na">_getter</span><span class="p">:</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">T</span><span class="p">;</span>
<span class="k">private</span> <span class="na">_value</span><span class="p">:</span> <span class="nx">T</span> <span class="o">|</span> <span class="k">typeof</span> <span class="nx">invalid</span> <span class="o">=</span> <span class="nx">invalid</span><span class="p">;</span>
<span class="kd">constructor</span><span class="p">(</span><span class="na">getter</span><span class="p">:</span> <span class="p">()</span> <span class="o">=></span> <span class="nx">T</span><span class="p">)</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">_getter</span> <span class="o">=</span> <span class="nx">getter</span><span class="p">;</span>
<span class="p">}</span>
<span class="kd">get</span><span class="p">()</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="k">this</span><span class="p">.</span><span class="nx">_value</span> <span class="o">===</span> <span class="nx">invalid</span><span class="p">)</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">_value</span> <span class="o">=</span> <span class="k">this</span><span class="p">.</span><span class="nx">_getter</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">return</span> <span class="k">this</span><span class="p">.</span><span class="nx">_value</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">invalidate</span><span class="p">()</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">_value</span> <span class="o">=</span> <span class="nx">invalid</span><span class="p">;</span>
<span class="c1">// TODO: We need some way to notify listeners here...</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is, of course, a naive skeleton of an implementation. Here are some things
to consider when implementing a full pull-based reactivity primitive:</p>
<ul>
<li>In <code class="language-plaintext highlighter-rouge">invalidate()</code> we need some way to notify our listeners that the value has
changed. In React and Rust futures there is only ever one listener. React is
the listener for a React component and the calling task of <code class="language-plaintext highlighter-rouge">poll()</code> is the
sole listener for Rust futures. We could allow our primitive to have many
listeners like an observable. Is there any efficiency advantage to limiting
the primitive to only one listener?</li>
<li>We need to be able to implement combinators on top of this primitive like
those in RxJS. Such as <code class="language-plaintext highlighter-rouge">map()</code> or <code class="language-plaintext highlighter-rouge">then()</code>. It’s fairly easy to imagine how
these would be implemented on top of this primitive.</li>
<li>What would an asynchronous version of this primitive look like? Should calling
<code class="language-plaintext highlighter-rouge">invalidate()</code> cancel the asynchronous work? Should there be another
cancellation mechanism?</li>
<li>There’s also the need for some kind of transaction system to avoid <a href="https://en.wikipedia.org/wiki/Reactive_programming#Glitches">reactive
glitches</a>. React, at a minimum, always defers state updates
to the next turn of the event loop. Should invalidations in our primitive work
this way as well?</li>
</ul>
<h2 id="further-reading">Further Reading</h2>
<ul>
<li><a href="https://github.com/ds300/derivablejs">Derivable.js</a> is one great
implementation of pull-based reactivity with a priority placed on performance.
However, the reactor system pushes new values to its callback which defeats
the purpose of pull-based reactivity. You aren’t able to schedule with low
priority the computation of the derivable value since the computation is
forced before your reactor callback is executed.</li>
<li><a href="https://github.com/skiplang/skip">Skip</a> is an experimental programming
language which is entirely reactive except for its external data sources which
use this pull-based reactivity model. When a Skip data source invalidates,
then Skip will invalidate all reactive cache entries that depend on that data
source.</li>
</ul>Observables, like those provided by Reactive Extensions, are a push based reactivity primitive. React, on the other hand, uses a pull based reactivity model. I’ll be using JavaScript and RxJS for my examples.