For our Intern Spotlight series, we’ll showcase the work of a summer intern at Shoreline with a technical deep dive. Jainam Shah is a CS master’s student at Columbia. This summer, his work focused on our WebCLI and Ops Notebook experiences.
A DevOps Notebook
SREs and DevOps love the command line and we do too! For example, let’s say you need to find and repair all disks with bad sectors. With Shoreline, you can filter down your hosts to only those with an error in the kernel log and then take that very same group and run your disk replacement script against them.
While the CLI is awesome for powerful one-liners, it’s not as useful for exploration and sharing. To find new patterns, we want to visually inspect lots of data quickly by scrolling through our metrics. After a debug session, we want to preserve what we have done and share it with others. Both of these are challenging for the tabular, single node view of a CLI.
Data science used to be in a similar place, but that all changed with the Jupyter Notebook. Jupyter Notebooks improved data science by making it interactive and visual. Data scientists could now run their commands and immediately see visible results. Afterward, they can share their analysis as a notebook file easily with others. With a web-based UI, getting started with Jupyter was easy.
Shoreline’s Notebooks bring these benefits to SRE and DevOps. An engineer can now run the Op language directly from within their browser. Like a notebook, metrics data is automatically visualized in a grid allowing resources to be inspected side by side. After a debug session, notebooks can be saved and then shared with the team.
My Project
I joined Shoreline hoping to get an opportunity to apply everything I have learned so far to solve real-world problems with interesting solutions while also growing in the process.
Interactive web experiences are my focus: I’ve built multiple single-page applications in React. From the outset, I knew that I wanted to develop an immersive web frontend for Shoreline.
While there already was a full web dashboard, there wasn’t a fluid way to interact with the underlying system. Shoreline has a particularly expressive interface: the Op language. Languages are as configurable as it gets - users are free to design their own experience by combining the different atoms in the language into complex statements. Before this project, when users wanted to use the Op language to control Shoreline, they needed to use the CLI.
The goal for my internship was to design and develop a fully immersive language experience for the web. In the end, I developed an interactive, web-based command-line tool along with an Ops notebook experience - like a Jupyter notebook, but for SREs and DevOps!
The Web CLI
The Shoreline CLI is a powerful tool for DevOps. Instead of SSHing into box after box of clusters, it enables executing commands simultaneously over multiple clusters. However, there are still some things like pagination and visualizations that are more difficult to achieve or awkward in a CLI based application.
So I wondered: How can I leverage the power of the browser to improve this experience? Ultimately, I decided to replicate the command line feel in the browser, emphasizing visualization and interaction to make it even more useful for end-users.
The CLI is expressive and lightweight, but the browser can visualize your data. From the CLI, users query their resources, inspect their metrics, and take action on their infrastructure. I realized that the output of CLI commands falls into four forms: text responses, resources, metric-query responses and linux command results.
For each request, I processed the request format and determined the appropriate output type to show the user. For queries like creating a resource, or getting a resource count, I displayed a text response with appropriate highlighting of the keywords. For resource queries, I showed a table with the different attributes for each resource and with support for smooth pagination. For metric queries, I showed a table just like the CLI, but I also offered the user an option to toggle to a chart view for a better experience. For linux command queries, I showed the stdout (or stderr) as expandable blocks of formatted text. This implementation improves the CLI experience by using the capabilities of the browser.
I worked as part of the UI team to build the WebCLI with TypeScript, React, and Material-UI. The application supports interactive outputs (for resource queries, metric queries, and Linux commands), clear visualizations, downloadable results, and more.
Shoreline Notebooks
Jupyter notebooks revolutionized data science by creating an interactive, exploratory, and shareable programming environment for data scientists. Instead of running their programs in batches, data scientists can iteratively examine data and computations to understand the patterns. They can then save their work like a “lab notebook,” easily sharing it with others to convey results and the steps needed to get there.
How can we bring this experience to DevOps? Well, the second part of my internship focused on just that: building a Jupyter notebook-like application for DevOps. But it was built on top of all my prior work: I was able to reuse the WebCLI component as an executable cell.
Like Jupyter, the Shoreline Notebook supports adding and deleting cells and saving cell outputs, bringing the exploratory experience to DevOps. It also allows creating, updating, deleting, and exporting multiple notebooks. SREs can share not only their results but their process for debugging.
The best part about this application is that no installation is required on the user’s end as it runs directly in the browser. In the future, Shoreline Notebooks will ultimately serve as a base platform for storytelling and Correction of Error documents. Entire runbook workflows can be ported to runnable notebooks and converted into automated remediation with one click.
During week eight of my internship, I demonstrated the project I had built in front of all the employees and received direct feedback. Everyone’s appreciation made me feel like my work had really made a difference and would prove to be useful to end-users. We also scheduled a bug-bash for my project where many people volunteered to come together to find edge cases where the application gave unexpected results and even suggested improvements. This bug hunt proved extremely helpful, and I took all the feedback and made the application even more robust.
Community at Shoreline
As soon as I joined, my manager asked me to set up some time with all the key people in the company to get to know it better. These one-on-one calls were beneficial and we discussed everything from technical topics to common alma maters and favorite food places. I even got a chance to get career advice from the CEO himself, who has already had three successful startups.
The New Grad employees at Shoreline were also invaluable. They set up a lunch call every Wednesday to chat and even play games. Plus, they were always down to hop on a call if I needed help with anything or just wanted some advice.
My manager was the one who guided me throughout the summer. He always gave me useful feedback during our daily syncs. He never tried to micromanage me but instead gave me a lot of freedom, which gave me the confidence to experiment with different approaches and iterate rapidly. I was another employee instead of a new intern, which made me feel like part of the team!
Closing Thoughts
I have never met a group of people more passionate about and invested in what they are doing than those at Shoreline. Everyone is working on solving a problem that they have faced in the past. I think that this is what is driving Shoreline forward on its definite path to success.
The Web CLI and Shoreline Notebook experiences are releasing in Q4 of this year. I’m so excited to see these features go into general availability. I’m confident that the interactive debugging experience and storytelling capabilities will make on-call much better!
If this work interests you, we are hiring! Please reach out to charles@shoreline.io