Short Take: Review Paper or Next New Idea?

Should you concentrate on writing a review article on an area of study that you know well or move on to the next new idea of yours, perhaps even a small new idea? The allure of going on to the next new shiny toy … errrr idea, is strong. To me there is a valued place for review articles (variously also called perspective or survey articles) in our intellectual canon. These articles hold our hands and safely let us take our first tentative steps into a new research area. These have an equally important role in taking the experts in an area and showing where their blinders are or in enthusing a band of researchers to try to tame a daunting problem.

Ingredients for a Good One

All these wonderful fruits of review articles can come about if they follow some fairly obvious writing principles.

1. Show me how we got here. A chronological development of the most significant ideas is often useful as this shows how each step was taken, how each step improved on the previous state, and perhaps points out to the intrepid what trajectory to follow.
2. Cluster, not enumerate. It is useful to provide a clustering of the ideas into a few manageable clusters (say 3-5). Each cluster will encompass many published works but will have one unifying theme. We find it much easier to grasp, and retain, such higher-level clusters than a flat enumeration of seemingly disconnected prior works.
3. Show me both sides of the coin. For each cluster of related works, show me where they do well and what are their weak points. There are likely application scenarios, requirements, or system settings where this class of techniques shines, and correspondingly a set where it crawls. Explain these to me in your article. And in explaining this, you do not need to delve into the details of each work being surveyed … move to a higher level of abstraction.
4. Show me the numbers. A quantitative comparison of one representative work from each cluster can shed important insight into a problem area. This is rarely done because it is often tedious to do. There is rarely one application/benchmark/system where all the competing approaches have been evaluated, so the author of the review article is left with the daunting task of creating such a unifying scenario. This scenario must have enough nuances to it to bring out the pros and cons of the competing approaches. While hard to do, this is one of those things that raises a good review paper into a work of sublime beauty.
5. A clarion call. A good review paper should leave off with a clarion call to arms, showing the road ahead, the problems to be vanquished to realize the promise of the field. Better still, if this call is coupled with a reflection of current tools and techniques that can help us in taming these problems.

Show Me and I Will Follow

OK so enough with the abstract talk. Let me point you to three review papers that to me embody these principles. There is a tiny teeny bit of self promotion flagrantly thrown in there.

1. “Learning under concept drift: A review,” J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang. IEEE Transactions on Knowledge and Data Engineering, 31(12):2346–2363, 2018. [Side note: This helped me come up to speed in an area I knew little about and with the pluck of my collaborators, Sean and David, get a paper accepted to NeurIPS 2020.]
2. “Cloud Programming Simplified: A Berkeley View on Serverless Computing,” Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Menezes Carreira, Karl Krauth, Neeraja Yadwadkar, Joseph Gonzalez, Raluca Ada Popa, Ion Stoica, and David A. Patterson. University of California, Berkeley Technical Report No. UCB/EECS-2019-3, February 2019. [Side note: What a wonderful insightful article standing at the verge of when this technology started taking over the world.]
3. “Challenges in Firmware Re-Hosting and Analysis,” Christopher Wright, William A. Moeglein (Sandia National Lab), Saurabh Bagchi, Milind Kulkarni, and Abraham A. Clements (Sandia National Lab). Accepted to appear in the ACM Computing Surveys (CSUR), pp. 1-37, 2020.

In Conclusion

Our community has made a great move by encouraging review articles and providing forums for such papers (ACM CSUR, IEEE Internet of Things Journal, IEEE Security and Privacy Symposium Systematization of Knowledge papers, etc.). We need to continue to celebrate such papers (give awards to the best, consider them in tenure and promotion) and need to occasionally do the heavy lifting of writing one of them.