A few tools you can use to find those elusive bottlenecks in Shiny app performance, adding a dash of interactivity to a reactable table, and save yourself many hours of manual effort with Quarto parameterized reporting.
Episode Links
Episode Links
- This week's curator: Colin Fay - @[email protected] [@ColinFay]](https://twitter.com/ColinFay) (X/Twitter)
- Unveiling Bottlenecks (Part 2): A Deep Dive into Profiling Tools\
- Creating interactive tables with reactable
- Automating Quarto reports with parameters
- Entire issue available at rweekly.org/2024-W35
- {golem} 0.5.0 is now available
- Shiny Developer Series Episode 12: Reactlog with Barrett Schlerke https://shinydevseries.com/interview/ep012/
- The Coding Cats shop https://www.etsy.com/shop/thecodingcats/
- A visual journey through world exhibitions https://georgios.quarto.pub/a-visual-journey-through-world-exhibitions/
- Six ways to find better content on Mastodon https://www.infoworld.com/article/2338712/6-ways-to-find-better-content-on-mastodon.html
- Use the contact page at https://serve.podhome.fm/custompage/r-weekly-highlights/contact to send us your feedback
- R-Weekly Highlights on the Podcastindex.org - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby, and then head over to the R-Weekly Highlights podcast entry on the index.
- A new way to think about value: https://value4value.info
- Get in touch with us on social media
- Eric Nantz: @[email protected] (Mastodon) and @theRcast (X/Twitter)
- Mike Thomas: @mike[email protected] (Mastodon) and @mikeketchbrook (X/Twitter)
- Tails and the Music Maker - Picolescence - zircon - https://ocremix.org/remix/OCR02176
- The Amazon Session - Ducktales - Gux - https://ocremix.org/remix/OCR00402
[00:00:03]
Eric Nantz:
Hello, friends. We're back at episode 176 of the Our Weekly Highlights podcast. This is the weekly podcast where we talk about the latest great resources that have been shared every single week at rweekly.org. It is basically your one stop shop for curated art content made by the community and for the community. My name is Eric Nantz, and I'm delighted you join us from wherever you are around the world in your favorite listening device or wherever you're tuning in. And, well, once again, I am flying the plane solo here again this week. Mike is still heads down on his, urgent project that he's tidying up now. Hopefully, he'll be back next week, and we can hear all about his adventures. But, nonetheless, we got some great highlights to talk about today, just you and me. We have a little cozy session here. But, of course, the Our Weekly Project itself is driven by the community in the form of our curators.
And our curator this week was Colin Fay, the author and architect of all things Golem, which, by the way, congratulations. Go to Colin and his team, I think, are for the release of Golem version 0 dot 5 that just got released. We'll put a link to their announcement blog post in the episode show notes. But, nonetheless, he had tremendous help as always from our fellow, our weekly team members, and contributors like you around the world with your poll requests and other suggestions. Speaking of Shiny, have you been here before? If you create an application that's used by, say, more than just you, You get it done. You get it released. Everything seems fine, and then you get that user feedback.
This can be both a good thing and maybe not so good thing depending on your perspective. A lot of times in my experience, my feedback would be, you know what, Eric? The app looks good, but it's just kinda slow compared to some of the other ones I use. And I'm thinking to myself, boy, when did where did I go wrong? There wasn't anything completely obvious. Right? Well, sometimes that puts you in the situation of trying to get to the bottom of this because in the end, the user experience of your Shiny app is a critically important piece to, obviously, get your users on board and to make sure that the application is having a sustained life cycle, if you will.
So you wanna get to the bottom of this right, but you're not quite sure where to start because the term profiling may give you a little bit of fear and when you hear that, but no fear here. We got a great blog post that showcase a couple of different tools that you can utilize along with a new tool that I wasn't aware of to look at different aspects of your shiny ass performance. And in particular, we're gonna be talking about a blog post from the Appsilon blog that was authored by Harsh Verma, one of their data scientists and developers at Appsilon, where he is talking to us about the different ways that you can go about profiling that Shiny application that has given you maybe a little bit of performance fits, if you will.
There are a couple different perspectives that he highlights here. The first of which is trying to figure out just you made all those inputs, you know, the elements that you can use in your Shiny app to, like, you know, enter text, change the slider value, drop down menus, and whatnot that the user is interacting with. And, of course, those are probably going to feed into some type of outputs in your app, maybe a visualization, a table of data, which we'll hear more about later, and other, you know, output, maybe HTML widgets, for example. And in between, you probably have a set of what are called reactive objects that are helping kind of facilitate processing into objects that are gonna be continuously updated depending on, say, inputs that are changing or other factors that are going on on the client side.
And as you build up more of these, it can be pretty easy to lose track of just what is actually connected to what and just how much you might say chatter is happening between the what the user sees and what the back end is producing to get to the outputs that the user sees. And that's where the first tool that Harsh, highlights here is the react log. The react log is something I hold close and dear to my shiny heart as a wonderful way to visualize this dependency graph of your inputs, your downstream reactives that are feeding into the outputs of your application. And trust me when I say, maybe it starts simple, but as you build the complexity, especially when you add modules in the mix, this can be a lot of things to keep track of. So the react log is a widget that will display, interactive flowchart of sorts where you can basically run the application in a typical flow, and then you can stop that and then put the react log in focus where it's going to let you traverse through that previous session where you were clicking inputs and then generating those outputs. So you've got a great way to kind of stop the steps, so to speak, so you can hone in on them further, and it's got all the connections between these inputs, reactives, and outputs in a very clear diagram. Again, highly interactive as well, so you can zoom in on certain points.
You can also filter based on different names of inputs. You get the state of the inputs such as whether they're ready for something new, whether they're actually calculating, or the process of invalidation where now they know something's wrong in that particular upstream dependency or should say outdated, so it's got to do some recalculation and whatnot. So this is a great way for you to see maybe you have a lot of reactive processing in between an input and a downstream output. Maybe there are opportunities to refine that a bit. And really seeing that that graph in front of you is a wonderful way to highlight just how complex that application's dependency graph has actually become.
And, again, I hold the prof again, I hold the React log very close to my dev toolbox. And if you wanna hear more about react log itself over in this little, you know, few minute overview I gave you, you wanna tune in to a previous episode of the shiny developer series that we have linked to in the show notes where we sat down with the architect of the react log from the Shiny team, Barish Slurkey, for a really fun conversation about the enhancements he made to reactlog many years ago. Now that's great to kinda get the overall picture of your dependency layout in your application, But there's also the fact of, well, when you're actually doing the calculations, what's really going on in terms of resource usage and time spent?
That's where a professor is, another wonderful, package in the R community authored by Winston Chang, also a member of the shiny team. That's where it comes to play to really help you in these situations. It can be used for things things besides shiny apps, but it runs very nicely to shiny apps as well. The flame graph is a great way to see, you know, a snapshot of just the number of steps or functions that are actually being called in these different operations that you're running in your application and then seeing just how long they're taking because the the x axis, the horizontal axis of this flame graph is the time spent in this application to create or, you know, produce the result of that function call.
And this is where you can maybe see, perhaps there's a, say, a data processing step that's just taking a lot of time and you can hone in on these different bars of these different pieces of your app functionality and start to see just what's really happening here. It's a great way to, again, pinpoint the bottlenecks and figuring out just how complex some of these operations actually happen to be. So a lot of times, the time spent is probably where you're gonna most focus on, but, again, it will have other outputs in terms of memory usage and other, number metrics that might be helpful to you.
Now this isn't like an either or. I think they both complement each other quite well, and that's what Harsh, talks about in the middle of the post. There isn't really a thing about which one is better. It's more about these are giving you different perspectives on the complexity of your Shiny application from that dependency graph all to just what is actually happening when the code's being executed. And then his post concludes with a new tool, new to me for sure, called shiny dot TikTok. And this is not an R package. This is actually a JavaScript utility where you can simply include this into your application with simply a call to a JavaScript script that's linked to in the blog post as part of your way in Shiny where you can arbitrarily execute a JavaScript, you know, script file. You just feed in via the tags $script function call and feed in the actual, CDN link to this utility.
And then once you have that in your app, you can just run your app as usual, say, in your local setup, do much like what I mentioned in the React log, just go through a typical usage. And then to view the output, you're gonna go into your browser's developer tools or browser tools, depending on which browser you're using. This is often what I'm using to look at the CSS behind certain elements, especially, you know, trying to theme things like my quartile presentations or for Shiny apps or bslib or whatnot. But there's also a JavaScript console in this browser tools where you can type in JavaScript commands on the fly.
And so he's got a, you know, narrative here for which commands you can run here, such as showing all the measurements in this job that were produced by the JavaScript utility. You can download all these into a CSV if you wanna post process them into R or other languages, and you can get a nice summary report, also in HTML format, where you could look at that, you know, as a as a separate web page. And it's kinda like a hybrid of the flame graph from profit is, albeit it's, you know, a little more, you know, organized, a little more streamlined, you might say. But it's also got color coding based on what are the outputs, also for calculations that are happening on the server such as, like, you know, server side loading or whatnot, and then also custom message handlers, which you might be utilizing in your app. But that's an interesting way for you to kinda get a quick take on how long some of these processes are taking, kinda like emerging between the maybe not so much emerging between the React log and the prophys visualizer, but it's an interesting take on it none nonetheless.
And what's nice is you could use this in your production apps. I don't believe it's causing a performance hit, although I haven't tried it yet. So you may wanna, you know, try that out yourself before you put it into production right away, but you can most certainly use this in a development version of your application locally as a quick way to get that, you know, more streamlined profiling output. So as always on our weekly highlights, I learn something new every time I talk about these great stories, so I will be adding shiny TikTok to my, developer toolbox alongside you know, again, much love to the react log package, and profit is has always been very illuminating in my, shiny profiling adventures.
So excellent blog post by Harsh, and, yeah, really looking forward to playing with shiny TikTok after this. You know, I can recall, you know, I would say many years ago as I was first getting into the art community, you know, fresh into my day job and fresh off my dissertation. In the way back years of, say, 2007, 2008. Again, to date myself, I've done that too much on this podcast episode already. Nonetheless, one of the things that was a bit of a challenge at the time was finding a really neat way to create polished looking tables outside of the typical Latex type setup. Nothing against Latex, but, it gave me, quite a bit of fits in my grad school days.
You fast forward to the today, and we have what I call an embarrassment of riches with how you can create really polished, aesthetically pleasing, powerful tables, both in the static format but also in the interactive format. And I'm really a big fan of the interactive format, hence I was, you know, psyched to give a submission to the recent POSITABLE contest of an interactive table. And the package that drove much of my contest submission was reactable. And it was a thrill to see or I should say, to meet the author of reactable, Greg Wynne. And it was a thrill to actually meet the author of reactable, Greg Lynn, from posit at the recent positconf. I just, you know, thanked him many times for reactable. I just had a tremendous time using it. So reactable gives you, again, yet another great way to create a really nice looking table in R, and our tutorial to talk about this of how to change it from a static representation to a very interactive table is coming from Albert Rapp, who returns to the highlights once again. It's been a little bit, Albert, but we're great to have you back on here.
And he is a business analyst working on some really interesting AI and ML stuff, apparently. He's also producing great content on his YouTube channel, so you should check that out if you haven't already, but he wrote a recent blog post with an associated video tutorial on how you can create an interactive table of reactable with a few lines of code and some great examples throughout. You know, I I really love making tables in r. Some might say I am the table sometimes. That's a little inside joke for some of you. But, nonetheless, let's get right into it. Shall we? So as usual, let's find some fun data to put in this table. And even though it's not the focus of this tutorial, the GT pack is by Rich Yone. It has a great set of built in datasets, and, yes, it happens to be with maybe some of my favorite food, pizza, of course. So he's got, Albert sets up with the, a filter data set of this pizza place data set for the pizza sales from Hawaii.
Man, that's on my bucket list to go to someday, maybe someday. Nonetheless, he does a little processing. We got a nice little tidy table here with the the sales by month and quarter, and the revenue that's being generated at that particular time. So what's nice is we just wanna get started right away with this tidy dataset. You just feed it into the reactable function. Feed in that dataset, and you've got yourself a nice interactive already kind of interactive table because in the blog post, you see this in action. It shows by default the first 10 rows, and there are actually 12 rows, one for each month. And you'll see at the bottom a toggle already go to the next set of rows, but Albert didn't have to write any code for that. It was already in the table function by reactable. So you already got a little interactivity along the way, but this is just using the default column name. So, of course, you might wanna spice things up a bit. So the first step is to add better, you know, human, you might say, human label type column names.
Now this is where there is a contrast to the GT package where in GT, it very much follows follows a ggplot or piping kind of flow where you can iteratively modify or add to the elements of your table. With reactable, everything is basically done in one particular function call, but with parameters that have a lot of associated sub parameters or general structures inside. One of those is for the case of column labels, you wanna leverage the columns argument of reactable and feed in a list with each list element named to the particular column you wanna add an attribute to. So in this example he's adding the new names of these columns using what's called the caldef function, also part of reactable, where you can feed in different aspects that you want to tweak in this column, and in this case it's just tweaking the name of the column. So once you get the hang of that, very easy to now have nice nice looking labels in your table of column names.
Now that's just the table on its own. Maybe you wanna start thinking about how you wanna publish this or share this with the community, so you wanna add a title and subtitle. There are ways in Reactable. If you know CSS and JavaScript, you can add almost anything you want to these tables, but there has been another great package in the r community that I've often used. In fact, I use this for my table contest submission as well. It is a, you might say, a an extension to reactable called reactable formatter or FMTR. I'm just spelling it out formatter there. And this package is authored by Kyle Kuiva. Hopefully, I'm saying that right. But he, or he or she but reactable formatter has a handy set of functions.
2 of them are adding a title and adding a subtitle. And when you change the style, though, should you wish, whereas he's a or Robert is able to change, say, the font weight of the subtitle to not be bold, but to be a normal type weight of font. So we've got a nice title above the table. But if you look at the table in the post, you notice that the revenue column is not quite nice yet because it's in that typical decimal notation. But we're talking about money here in US dollars. That's what the data set's using. So in that cal def function, you you can change much more than just the name of a column.
You can also change the format of the cells, and that's using the cal format function where this could be anything from, like, a date or, in this case, currency, or even customizing a custom format of your own. So here, Albert just simply specifies as the USD for the currency, you know, type, and then to add separators, I e the comma the commas between each each three digit for larger revenue. And now you've got a table that has, you know, nicely formatted lined up values of revenue in that last column. Now as you look at this table, you notice that we know we don't just have month, we also have quarter as well.
But wouldn't it be nice to be able to let the user kinda toggle easy navigation between these groups? Maybe they're more interested in, say, the Q2 or the Q4 or whatnot. Well, reactable gives you a nice way to add groups, and, again, this could not be more simple in the reactable construct because there's a parameter called group by. You feed in the name of the column that you wanna use as your grouping, and right away, you'll see in the output from the blog post, you've got a nice collapsible element next to each quarter, a little caret arrow, if you will. You click on that, and now you've got the month sales and revenue showing up. And you can expand all of them, you can expand none of them. You get to choose, and now we have and now we have an even more interactive table for you to play with.
Looking nice here. Right? Well, maybe let's take it up a notch if we wanna share this with, say, a stakeholder or the community. You might want some nice, you know, summaries for each of these quarters as well. And what can we do for that? Well, what we can do there is we can say, okay. In the quarter view, if everything collapsed, what if we wanna add all the values under sales so that that displays when it's, like, nested, but when you unnest it, then you get the individual values right below it. So that's where you can now add an aggregate call to that call def of the sales column, and you can choose which aggregation method you wanna do such as the average or mean or in this case a summarization or sum.
So that's great. Now you can have for each group the total sales showing up in that default view with everything collapsed. Again, we're talking about interactive tables. Right? Well, what if you wanna instead of point and clicking, you wanna filter for a particular month? Well, reactable, again, has a handy little parameter in calldef called filter easy for me to say. Has a handy little parameter in the calldef called filterable where you enable that toggle to true, and then that particular column now has a nice little text box right under the column heading where you can type in the name of a particular, say, month you want, and it will auto complete, or I should say dynamically update based on the first few letters.
So I just start typing j a, I'll immediately get January showing up under the quarter quarter 1, you know, grouping. Again, very easy way to add filtering elements right away. You could have done that with sales as well, but in the end, this is a great great little showcase of even more interactivity without any custom CSS or JavaScript required. Now hold that thought for a second because we're gonna dive into that a little bit here, because what if at the very bottom of this table you want, you know, a common question to be answered of just what is the exact total of all the years revenue and all the years sales.
So there is a way to do this a couple of different ways in reactable, one of which is adding a table footer using our code to basically cuss define a custom function that takes as inputs the values for that particular column and the name of it, if you wish. If you want to do some of it, you can. And that's where you can use any ordinary r function. In this case, he's using the sum function again to summarize all to add up all those revenue values. And then from the scales package, he's making sure that's represented in the near currency format using the scales dollar function. Again, that's based in r. That's not based in reactables formatting.
And once you do that, yeah, it it does work, but there's a little nuance here. Let's say you wanna do that filtering after all. Maybe you wanna go to January instead using that annual text box filter. Well, you notice when you do that, that revenue summarization and that sales summarization does not change. It's not respecting that filter because that filter is being done in what we call client side interactions. The the summarization itself was computed ahead of time before the table was actually rendered. And unlike in Shiny, there's no, like, direct communication between what the user is doing on the filtering side and the underlying r function that created that summarization.
Well, have no fear, folks. If you're willing to invest in a little JavaScript knowledge here, you can have a dynamically updating summarization with JavaScript code. Now this is where, you know, if you're new to JavaScript, it'll it'll look maybe a little different at first. But once you get the hang of it, I think you can do little basic things like this without too much trouble. But the example blog post, of course, has the code where Albert is defining a custom JavaScript function that initializes an empty total of 0. And then basically for each row that's currently present using this state object, again, the state is like what JavaScript sees is what the user is seeing in that table.
So for example, if they're creating a filter, they're just gonna see what that state object is just gonna represent, what is actually showing on that table. Great. So he takes that and then for each of the rows that are in that state of the filter, he's gonna add up the values based on the column ID that's being supplied here and then return that total. So he does that for both the sales and the revenue along with some formatting of the revenue using some JavaScript, you know, functions as well, which, again, you can see in the show note I mean, in the blog post. And there you go. Now when you do that filtering, I'll even type it while I'm recording here.
Sure enough, when I filter for January, that total is now updating accordingly. So this is terrific. A terrific way to make sure that you get that shiny like experience, but again, this is all done in client side, and reactable has JavaScript bindings that you can customize as much or as little as you like. Lastly, speaking of making things look good, Albert wants to change the style of those, what I call, nested group rows versus the, you know, the rows of the actual data itself. His first attempt at this actually ends up coloring every row in the same color, which kind of defeats the purpose, doesn't it? But that's a artifact of using this row group this theme function with the reactable theme, function, and that, by proxy, is selecting every row for that theme. So, uh-oh, that's not quite good.
What can we do differently? Well, again, we can go in the JavaScript world where he defines a JavaScript function to say, you know what? I'm gonna take the row itself, which is called row info. It's like metadata for the rows each in each of the in each of the table. And for each of these groups, that first row, which is just showing the quarter name and whatnot, that's level 0. Remember, JavaScript index start at 0, not 1 like an r, so that might trip you up a bit. But he's saying if that level is 0 that's being shown, then do the custom background on the cell, you know, color.
Otherwise, leave the rest the heck alone. And sure enough, at the end product, you've got those quarter rows with that nice little, you know, shading so you can quickly visually distinguish those from the actual data cells as well. And last but not least, he does a little change up to the style of the footer, and this is again very simple to do. Again, a little bit of, CSS demonstrated here where you can add a footer style with custom CSS function from the HTML tools package, or you can give it, like, the font weight, and in this case a little line at the top called border top. So you can quickly see just like you might be writing this on paper, putting a line under the numbers you're calculating, and then making sure that visually pops out.
So again, a tour de force of how you can take a native reactable table that just comes with that data frame being fed into it, all the way to a very nice, polished, and again highly interactive, multiple levels, table for you to look at. Again, reactable and GT, wonderful packages for creating tables. Albert's been doing a lot of content on each of these, so you wanna check out the archive of his post for of his blog, I should say, for many of our posts that talk about using these these great packages in action. So I am thrilled thrilled to see this.
And rounding out our highlights today is another great showcase of another awesome piece of technology using a framework that's quite familiar to those that have been in the R Markdown ecosystem to save you a bunch of time and a lot of future effort. And, yeah, I think future you will thank you for investing in such techniques. We are specifically speaking about the Quarto publication system, which has been a hot topic ever since it was announced at Posiconf back earlier in 2022. I can't believe it's been that long, but, yeah, Quartle has been around a little bit already. And in particular, it has a lot of functionalities under the hood, but one of those which is definitely been inspired by the R Markdown predecessor is the idea of creating parameterized reports, in that you can compile a report and feed into it a set of values, I e parameters, to maybe update or dynamically change some content depending on those parameters.
And one of the real innovative thought leaders in this space of taking advantage of parameterized reporting from the Rmarkdown ecosystem and now Quartal is the author of this latest blog post on deposit blog, JD Ryan, who has had quite a career transition, I must say, as she has gone from being literally in the field or you might say in the streams working in the Washington State Department of Agriculture as an environmental technician, and now she is leveraging her data science skills more in the, I guess, an office type environment. But I had the great pleasure of seeing JD again at the aforementioned positconf earlier this month, and I must say for the many conversations we had, whether it was at the data science hangout or, you know, here pepping me up for my talk or even afterwards as the conference is winding down.
Yeah. Her, her energy is infectious and, sometimes I wonder where did I get that energy? Where can I get that? That fountain of youth that she seems to be drinking from? But nonetheless, it was great to touch base with her and and yes, she was kind enough to give me a little sticker too. And by the way, if you want some awesome swag, if you're a cats lover like she and I are and you want to flaunt R at the same time, you want to check out her awesome little shop on Etsy, which I'll link to in the show notes. Nonetheless, JD has been in this, world of creating privatized reports for quite some time, and she mentions how much it's been extremely helpful as she was transitioning to her data science role in terms of creating in upwards of hundreds of these reports for farmers across the area based on their particular data needs.
And instead of hard coding all this 1 by 1 for, like I said, over 300 of these, she was able to leverage the parameter equals the parameter functionality to create both interactive and static or PDF versions of these reports and have them dynamically render in a batch setting. So how do you actually get there? That's where the meat of the post dives in here, where there are different kind of high level goals that you want to look at depending on how far you want to take it. Obviously, if you only have a few reports, maybe maybe manual is the better ever for you, and that might be might be what you're used to. But then, as like I said, the scale starts to escalate.
If you find yourself regenerating a lot of these things manually, that's when you're on and start investing into the idea of parameterized reports. So how do they actually work? Well, no matter if you're doing a report based in r or based in Python, you will have a capacity to be able to specify these parameters either in the front matter YAML of that quartile report if you're leveraging the knitter engine, I. E. Using r, or if you're in the Jupyter setting, if you're leveraging Python, you can do a specific code chunk with a optional, tag attribute called parameters, and you can literally just type in the values 1 by 1.
Obviously, the parameter values are more optimized for, you know, textual type parameters, maybe numbers. You're probably not going to be able to use complex classes because it's got to be something that can be specified in text, by the renderer. But once you have that going and once you have those parameters defined, then you have many different ways to actually compile this. First of which is within a IDE like Rstudio or leveraging the quartile extension in Versus Code or positron, you can compile the report and it will use the default values of those parameters that you specified so you can get a feel for if the parameters are working as expected.
And so another option is maybe you want to keep those default values as is, but you wanna experiment with changing maybe 1 or 2 of them. Well, quartile is not like a r package, like rmarkdown was. It is actually a utility, a software utility on its own with its own command line interface. So you can run the quartal render function and feed in the quartal markdown document, you might call it template, and then you can specify 1 or more parameter values with the dash capital p flag, and then give the name of the parameter, a colon, and then that parameter value. So that's a great way if you don't want to change your your default parameters and you want to get a quick take on maybe changing those parameters on the fly, you can quickly do that.
Or if you don't want to type those command line flags, you could have a separate YAML file called params. Yaml, where then you can simply put them much like you would in that frontliner. Yaml for the default parameter settings, and just in the Kortal renderer call use the dash dash execute dash params flag, feed in that YAML, and now you've got again that that report compiled using those new values. And there is more, of course. If you are in the R ecosystem, there is the quartal R package, which again is not the quartal engine itself, it's just you can think of it as like an API type package to the quartal static rendering engine, where then that has an execute underscore params argument, and you can feed the parameters as a list object, and then feeding it as input the name of the file that you're going to render.
So you could do that from R itself, and now the wheels are probably turning here. You got all these different ways of executing and compiling the report with parameters. What are ways you can actually automate that? Well, there are a few techniques here and I've used a couple of these already, but they're they're really, really solid. One of which is to have a data frame which has all the combinations of the parameters that you're interested in rendering as each row. So in her example, she's got a year, a producer ID, and literally does a expand grid of these 2 year values and 4 ID values, and you'll get then this, 8 row data frame with each of those parameter combinations.
That's pretty straightforward, and what you do with that is you can augment then the different output formats you want. Maybe it's HTML, and then the actual file name using a little paste magic, and then you can have then a nested column that has these parameter values as lists, I. E. The columns that are already in this data frame, and make it a named list. And she's got a nice little snippet of code that does this with the purrr package, another one I simply can't live without, and again, if you love your cats, then how can you not love the purr package? So it seems like fit for purpose here. Then once you have that, then you can have a customized tidy kind of data frame organized by 1 row per iteration, and then back to purr you can use the handy function called p walk which you can feed in the data frame itself and then that basically means that each column of the data frame is like a function parameter That can get fed into the quartal render function, and those parameters are going to be fed very clearly, as named arguments.
So again this may take a little getting used to, but again you can leverage her snippets of code that she has in the post and try it out for yourself, and this can be part of a larger scale pipeline. Just imagine that you want to do these on a routine basis, you know, you can find ways to automate this. Heck, you you know, quarto is a first class citizen output format of the targets package that I speak so highly about. You could lever something like this in your targets reports as well. There are lots of interesting ways you can use this, technique, again, to save you a boatload of time and effort in the future.
So JD definitely has a lot more materials than what this blog post has here, and she's done a workshop about this in the past. She's done a presentation at Pazitconf, last year about this workflow and how it applied to her daily work. So, yeah, you'll be invited to check those out as well And again, love love the cat pics in this. I got it. Yeah. Makes me miss my cats from the yesteryear. But nonetheless, really great blog post by JD here. You can tell it's got all the bells and whistles of quarto itself in the blog post because it got nice little tabbed interface to go from the r snippet for that automated execution to a bash scripting, which links to another blog post by Solomon Moon who talks about how you can do this in a bash script. So again, whatever your flavor is, you've she's got you covered with different examples here.
And if that wasn't enough, boy, this issue has got a whole bunch in here that we wish we could cover today, and on in its entirety, but, yeah, time's not unlimited, unfortunately. But you can find all the additional resources at roku.org, and we'll take a minute here to talk about one of my additional finds that I think, might be worth looking at as you listen to the show and afterwards. And we heard announced at Posicomp in one of the talks a wonderful new extension for Quartle called Close Read. And what this is giving you is a way to have that kind of scrolly telling functionality in your quarter report that you often see in these websites that are kind of this hybrid of infographics and other, you know, neat neat utilities.
And the way you scroll through it, it kinda updates the content dynamically based on where you're at. It maybe freezes certain elements. Well, this this extension, Close Read, is really really great in this space. We haven't really had any we've had attempts at this before in the art community, but I think this one really nails it. So this, link is from Georgios Cameronis. I believe he also gave a talk at Posikov and this is a visual journey through world exhibitions and you kinda have to see it to believe it. I can't do enough, you know, enough justice on the audio form, but it's got this great interactive map that depending on where you're going in the story, it's gonna zoom in on different regions wherein it talks about the narrative behind these different exhibits.
It'll splice in some authentic pictures from these exhibits, but it's a great showcase of what is possible with close read and I am definitely going to take a look at this as I think about content that can be engaging to different audiences. I mean, gosh, my wish is I could do something like this in life sciences. I'm not sure if that type of material would be fit for it. Hey, you never know. Maybe so. But maybe I could find some other uses for it. But, Close Read is still early days, but it looks like a few people are already putting it through the paces. In fact, I saw another contributor to our weekly, highlights in the past, Nicole Raney has also done an example of her tidy Tuesday visualization in a close read document as well. We'll have a link to that in the show notes. So starting to see it in the wild, and I'm definitely interested in seeing where the uses of that extension end up going.
And yeah, like I said, there's a lot more to this issue, but we're gonna have to wrap things up here as as we're winding down. But of course, I like to always close with telling you how you can get involved to help the project, as well as this very podcast. First, where can you get involved with our weekly? Well, again, visit our weekly dotorg, and we welcome all your contributions to that great new blog post showcasing R and Data Science, maybe a great new package that you discovered or something that's had a new release, a great workshop tutorial, or maybe another podcast or videos out there.
Either way, there are different categories for each of these type of content, and the best way to do that is via a pull request already linked in the top right corner in that handy little ribbon on the Our Weekly site. You click that, you'll be taken to a predefined template. You'll fill out a little bit of metadata, and all we need is a link, all marked down all the time. Just like when you're writing that fancy quartile parameterized report, you use a markdown for that too. So we invite you to share your resources on there and the curator of the week will be glad to merge that in. And also, we are definitely open for a curator spots as well. We would love to hear from you on that front.
We have details on the process at rweekly.org, and that will take you to the GitHub repo where you have a nice reading that's been put together about how to get involved on the curation front. And also we'd love to hear from you on this very podcast, so we have a little contact page directly in the show notes that you can you can use to quickly send myself and Mike a message. You can also, if you're on a modern podcast app, some of my favorites are Podverse and Fountain. Also for iOS users, Casa Mac, I hear great things about. With one of those apps, you can send us a fun little boost along the way that goes directly to us on the on the podcast team and no middle party involved.
So that's available as well, but also you can find me on these social medias out there. Mostly on Mastodon these days at our podcast at podcast index.social. And I did, have a good friend of mine on one of the other communities I'm part of trying to figure out the best way to get started with Mastodon. If that is something you as well are trying to get into, I'll put a link to a couple, great write ups that I found helpful in the community. If you're new to your Mastodon journey, because I'm starting to see a lot of my friends from data science and the art community are starting to get on Mastodon, but the key is finding the best way to get in touch with them so you can find their content. So I have some resources about that in the show notes of this episode.
You can also find me on LinkedIn as well. Just search for my name and you'll find me on there and somewhat sporadically on the weapon x thingy with at the r cast. Well, that'll do it for this episode of Rookery Highways. Can't believe we're at episode 176 already. Gosh. We're 200 is not that far away, so hopefully we'll get there. One way or another, I hope. But again, next week I hope to have my trusty companion here, Mike, back on the mic with me, so that will be, you know, a welcome change instead of just hearing me blab all by myself. In any event, we're gonna close-up shop here that will wrap up episode 176 of our weekly highlights and we'll be back with a new episode next week.
Hello, friends. We're back at episode 176 of the Our Weekly Highlights podcast. This is the weekly podcast where we talk about the latest great resources that have been shared every single week at rweekly.org. It is basically your one stop shop for curated art content made by the community and for the community. My name is Eric Nantz, and I'm delighted you join us from wherever you are around the world in your favorite listening device or wherever you're tuning in. And, well, once again, I am flying the plane solo here again this week. Mike is still heads down on his, urgent project that he's tidying up now. Hopefully, he'll be back next week, and we can hear all about his adventures. But, nonetheless, we got some great highlights to talk about today, just you and me. We have a little cozy session here. But, of course, the Our Weekly Project itself is driven by the community in the form of our curators.
And our curator this week was Colin Fay, the author and architect of all things Golem, which, by the way, congratulations. Go to Colin and his team, I think, are for the release of Golem version 0 dot 5 that just got released. We'll put a link to their announcement blog post in the episode show notes. But, nonetheless, he had tremendous help as always from our fellow, our weekly team members, and contributors like you around the world with your poll requests and other suggestions. Speaking of Shiny, have you been here before? If you create an application that's used by, say, more than just you, You get it done. You get it released. Everything seems fine, and then you get that user feedback.
This can be both a good thing and maybe not so good thing depending on your perspective. A lot of times in my experience, my feedback would be, you know what, Eric? The app looks good, but it's just kinda slow compared to some of the other ones I use. And I'm thinking to myself, boy, when did where did I go wrong? There wasn't anything completely obvious. Right? Well, sometimes that puts you in the situation of trying to get to the bottom of this because in the end, the user experience of your Shiny app is a critically important piece to, obviously, get your users on board and to make sure that the application is having a sustained life cycle, if you will.
So you wanna get to the bottom of this right, but you're not quite sure where to start because the term profiling may give you a little bit of fear and when you hear that, but no fear here. We got a great blog post that showcase a couple of different tools that you can utilize along with a new tool that I wasn't aware of to look at different aspects of your shiny ass performance. And in particular, we're gonna be talking about a blog post from the Appsilon blog that was authored by Harsh Verma, one of their data scientists and developers at Appsilon, where he is talking to us about the different ways that you can go about profiling that Shiny application that has given you maybe a little bit of performance fits, if you will.
There are a couple different perspectives that he highlights here. The first of which is trying to figure out just you made all those inputs, you know, the elements that you can use in your Shiny app to, like, you know, enter text, change the slider value, drop down menus, and whatnot that the user is interacting with. And, of course, those are probably going to feed into some type of outputs in your app, maybe a visualization, a table of data, which we'll hear more about later, and other, you know, output, maybe HTML widgets, for example. And in between, you probably have a set of what are called reactive objects that are helping kind of facilitate processing into objects that are gonna be continuously updated depending on, say, inputs that are changing or other factors that are going on on the client side.
And as you build up more of these, it can be pretty easy to lose track of just what is actually connected to what and just how much you might say chatter is happening between the what the user sees and what the back end is producing to get to the outputs that the user sees. And that's where the first tool that Harsh, highlights here is the react log. The react log is something I hold close and dear to my shiny heart as a wonderful way to visualize this dependency graph of your inputs, your downstream reactives that are feeding into the outputs of your application. And trust me when I say, maybe it starts simple, but as you build the complexity, especially when you add modules in the mix, this can be a lot of things to keep track of. So the react log is a widget that will display, interactive flowchart of sorts where you can basically run the application in a typical flow, and then you can stop that and then put the react log in focus where it's going to let you traverse through that previous session where you were clicking inputs and then generating those outputs. So you've got a great way to kind of stop the steps, so to speak, so you can hone in on them further, and it's got all the connections between these inputs, reactives, and outputs in a very clear diagram. Again, highly interactive as well, so you can zoom in on certain points.
You can also filter based on different names of inputs. You get the state of the inputs such as whether they're ready for something new, whether they're actually calculating, or the process of invalidation where now they know something's wrong in that particular upstream dependency or should say outdated, so it's got to do some recalculation and whatnot. So this is a great way for you to see maybe you have a lot of reactive processing in between an input and a downstream output. Maybe there are opportunities to refine that a bit. And really seeing that that graph in front of you is a wonderful way to highlight just how complex that application's dependency graph has actually become.
And, again, I hold the prof again, I hold the React log very close to my dev toolbox. And if you wanna hear more about react log itself over in this little, you know, few minute overview I gave you, you wanna tune in to a previous episode of the shiny developer series that we have linked to in the show notes where we sat down with the architect of the react log from the Shiny team, Barish Slurkey, for a really fun conversation about the enhancements he made to reactlog many years ago. Now that's great to kinda get the overall picture of your dependency layout in your application, But there's also the fact of, well, when you're actually doing the calculations, what's really going on in terms of resource usage and time spent?
That's where a professor is, another wonderful, package in the R community authored by Winston Chang, also a member of the shiny team. That's where it comes to play to really help you in these situations. It can be used for things things besides shiny apps, but it runs very nicely to shiny apps as well. The flame graph is a great way to see, you know, a snapshot of just the number of steps or functions that are actually being called in these different operations that you're running in your application and then seeing just how long they're taking because the the x axis, the horizontal axis of this flame graph is the time spent in this application to create or, you know, produce the result of that function call.
And this is where you can maybe see, perhaps there's a, say, a data processing step that's just taking a lot of time and you can hone in on these different bars of these different pieces of your app functionality and start to see just what's really happening here. It's a great way to, again, pinpoint the bottlenecks and figuring out just how complex some of these operations actually happen to be. So a lot of times, the time spent is probably where you're gonna most focus on, but, again, it will have other outputs in terms of memory usage and other, number metrics that might be helpful to you.
Now this isn't like an either or. I think they both complement each other quite well, and that's what Harsh, talks about in the middle of the post. There isn't really a thing about which one is better. It's more about these are giving you different perspectives on the complexity of your Shiny application from that dependency graph all to just what is actually happening when the code's being executed. And then his post concludes with a new tool, new to me for sure, called shiny dot TikTok. And this is not an R package. This is actually a JavaScript utility where you can simply include this into your application with simply a call to a JavaScript script that's linked to in the blog post as part of your way in Shiny where you can arbitrarily execute a JavaScript, you know, script file. You just feed in via the tags $script function call and feed in the actual, CDN link to this utility.
And then once you have that in your app, you can just run your app as usual, say, in your local setup, do much like what I mentioned in the React log, just go through a typical usage. And then to view the output, you're gonna go into your browser's developer tools or browser tools, depending on which browser you're using. This is often what I'm using to look at the CSS behind certain elements, especially, you know, trying to theme things like my quartile presentations or for Shiny apps or bslib or whatnot. But there's also a JavaScript console in this browser tools where you can type in JavaScript commands on the fly.
And so he's got a, you know, narrative here for which commands you can run here, such as showing all the measurements in this job that were produced by the JavaScript utility. You can download all these into a CSV if you wanna post process them into R or other languages, and you can get a nice summary report, also in HTML format, where you could look at that, you know, as a as a separate web page. And it's kinda like a hybrid of the flame graph from profit is, albeit it's, you know, a little more, you know, organized, a little more streamlined, you might say. But it's also got color coding based on what are the outputs, also for calculations that are happening on the server such as, like, you know, server side loading or whatnot, and then also custom message handlers, which you might be utilizing in your app. But that's an interesting way for you to kinda get a quick take on how long some of these processes are taking, kinda like emerging between the maybe not so much emerging between the React log and the prophys visualizer, but it's an interesting take on it none nonetheless.
And what's nice is you could use this in your production apps. I don't believe it's causing a performance hit, although I haven't tried it yet. So you may wanna, you know, try that out yourself before you put it into production right away, but you can most certainly use this in a development version of your application locally as a quick way to get that, you know, more streamlined profiling output. So as always on our weekly highlights, I learn something new every time I talk about these great stories, so I will be adding shiny TikTok to my, developer toolbox alongside you know, again, much love to the react log package, and profit is has always been very illuminating in my, shiny profiling adventures.
So excellent blog post by Harsh, and, yeah, really looking forward to playing with shiny TikTok after this. You know, I can recall, you know, I would say many years ago as I was first getting into the art community, you know, fresh into my day job and fresh off my dissertation. In the way back years of, say, 2007, 2008. Again, to date myself, I've done that too much on this podcast episode already. Nonetheless, one of the things that was a bit of a challenge at the time was finding a really neat way to create polished looking tables outside of the typical Latex type setup. Nothing against Latex, but, it gave me, quite a bit of fits in my grad school days.
You fast forward to the today, and we have what I call an embarrassment of riches with how you can create really polished, aesthetically pleasing, powerful tables, both in the static format but also in the interactive format. And I'm really a big fan of the interactive format, hence I was, you know, psyched to give a submission to the recent POSITABLE contest of an interactive table. And the package that drove much of my contest submission was reactable. And it was a thrill to see or I should say, to meet the author of reactable, Greg Wynne. And it was a thrill to actually meet the author of reactable, Greg Lynn, from posit at the recent positconf. I just, you know, thanked him many times for reactable. I just had a tremendous time using it. So reactable gives you, again, yet another great way to create a really nice looking table in R, and our tutorial to talk about this of how to change it from a static representation to a very interactive table is coming from Albert Rapp, who returns to the highlights once again. It's been a little bit, Albert, but we're great to have you back on here.
And he is a business analyst working on some really interesting AI and ML stuff, apparently. He's also producing great content on his YouTube channel, so you should check that out if you haven't already, but he wrote a recent blog post with an associated video tutorial on how you can create an interactive table of reactable with a few lines of code and some great examples throughout. You know, I I really love making tables in r. Some might say I am the table sometimes. That's a little inside joke for some of you. But, nonetheless, let's get right into it. Shall we? So as usual, let's find some fun data to put in this table. And even though it's not the focus of this tutorial, the GT pack is by Rich Yone. It has a great set of built in datasets, and, yes, it happens to be with maybe some of my favorite food, pizza, of course. So he's got, Albert sets up with the, a filter data set of this pizza place data set for the pizza sales from Hawaii.
Man, that's on my bucket list to go to someday, maybe someday. Nonetheless, he does a little processing. We got a nice little tidy table here with the the sales by month and quarter, and the revenue that's being generated at that particular time. So what's nice is we just wanna get started right away with this tidy dataset. You just feed it into the reactable function. Feed in that dataset, and you've got yourself a nice interactive already kind of interactive table because in the blog post, you see this in action. It shows by default the first 10 rows, and there are actually 12 rows, one for each month. And you'll see at the bottom a toggle already go to the next set of rows, but Albert didn't have to write any code for that. It was already in the table function by reactable. So you already got a little interactivity along the way, but this is just using the default column name. So, of course, you might wanna spice things up a bit. So the first step is to add better, you know, human, you might say, human label type column names.
Now this is where there is a contrast to the GT package where in GT, it very much follows follows a ggplot or piping kind of flow where you can iteratively modify or add to the elements of your table. With reactable, everything is basically done in one particular function call, but with parameters that have a lot of associated sub parameters or general structures inside. One of those is for the case of column labels, you wanna leverage the columns argument of reactable and feed in a list with each list element named to the particular column you wanna add an attribute to. So in this example he's adding the new names of these columns using what's called the caldef function, also part of reactable, where you can feed in different aspects that you want to tweak in this column, and in this case it's just tweaking the name of the column. So once you get the hang of that, very easy to now have nice nice looking labels in your table of column names.
Now that's just the table on its own. Maybe you wanna start thinking about how you wanna publish this or share this with the community, so you wanna add a title and subtitle. There are ways in Reactable. If you know CSS and JavaScript, you can add almost anything you want to these tables, but there has been another great package in the r community that I've often used. In fact, I use this for my table contest submission as well. It is a, you might say, a an extension to reactable called reactable formatter or FMTR. I'm just spelling it out formatter there. And this package is authored by Kyle Kuiva. Hopefully, I'm saying that right. But he, or he or she but reactable formatter has a handy set of functions.
2 of them are adding a title and adding a subtitle. And when you change the style, though, should you wish, whereas he's a or Robert is able to change, say, the font weight of the subtitle to not be bold, but to be a normal type weight of font. So we've got a nice title above the table. But if you look at the table in the post, you notice that the revenue column is not quite nice yet because it's in that typical decimal notation. But we're talking about money here in US dollars. That's what the data set's using. So in that cal def function, you you can change much more than just the name of a column.
You can also change the format of the cells, and that's using the cal format function where this could be anything from, like, a date or, in this case, currency, or even customizing a custom format of your own. So here, Albert just simply specifies as the USD for the currency, you know, type, and then to add separators, I e the comma the commas between each each three digit for larger revenue. And now you've got a table that has, you know, nicely formatted lined up values of revenue in that last column. Now as you look at this table, you notice that we know we don't just have month, we also have quarter as well.
But wouldn't it be nice to be able to let the user kinda toggle easy navigation between these groups? Maybe they're more interested in, say, the Q2 or the Q4 or whatnot. Well, reactable gives you a nice way to add groups, and, again, this could not be more simple in the reactable construct because there's a parameter called group by. You feed in the name of the column that you wanna use as your grouping, and right away, you'll see in the output from the blog post, you've got a nice collapsible element next to each quarter, a little caret arrow, if you will. You click on that, and now you've got the month sales and revenue showing up. And you can expand all of them, you can expand none of them. You get to choose, and now we have and now we have an even more interactive table for you to play with.
Looking nice here. Right? Well, maybe let's take it up a notch if we wanna share this with, say, a stakeholder or the community. You might want some nice, you know, summaries for each of these quarters as well. And what can we do for that? Well, what we can do there is we can say, okay. In the quarter view, if everything collapsed, what if we wanna add all the values under sales so that that displays when it's, like, nested, but when you unnest it, then you get the individual values right below it. So that's where you can now add an aggregate call to that call def of the sales column, and you can choose which aggregation method you wanna do such as the average or mean or in this case a summarization or sum.
So that's great. Now you can have for each group the total sales showing up in that default view with everything collapsed. Again, we're talking about interactive tables. Right? Well, what if you wanna instead of point and clicking, you wanna filter for a particular month? Well, reactable, again, has a handy little parameter in calldef called filter easy for me to say. Has a handy little parameter in the calldef called filterable where you enable that toggle to true, and then that particular column now has a nice little text box right under the column heading where you can type in the name of a particular, say, month you want, and it will auto complete, or I should say dynamically update based on the first few letters.
So I just start typing j a, I'll immediately get January showing up under the quarter quarter 1, you know, grouping. Again, very easy way to add filtering elements right away. You could have done that with sales as well, but in the end, this is a great great little showcase of even more interactivity without any custom CSS or JavaScript required. Now hold that thought for a second because we're gonna dive into that a little bit here, because what if at the very bottom of this table you want, you know, a common question to be answered of just what is the exact total of all the years revenue and all the years sales.
So there is a way to do this a couple of different ways in reactable, one of which is adding a table footer using our code to basically cuss define a custom function that takes as inputs the values for that particular column and the name of it, if you wish. If you want to do some of it, you can. And that's where you can use any ordinary r function. In this case, he's using the sum function again to summarize all to add up all those revenue values. And then from the scales package, he's making sure that's represented in the near currency format using the scales dollar function. Again, that's based in r. That's not based in reactables formatting.
And once you do that, yeah, it it does work, but there's a little nuance here. Let's say you wanna do that filtering after all. Maybe you wanna go to January instead using that annual text box filter. Well, you notice when you do that, that revenue summarization and that sales summarization does not change. It's not respecting that filter because that filter is being done in what we call client side interactions. The the summarization itself was computed ahead of time before the table was actually rendered. And unlike in Shiny, there's no, like, direct communication between what the user is doing on the filtering side and the underlying r function that created that summarization.
Well, have no fear, folks. If you're willing to invest in a little JavaScript knowledge here, you can have a dynamically updating summarization with JavaScript code. Now this is where, you know, if you're new to JavaScript, it'll it'll look maybe a little different at first. But once you get the hang of it, I think you can do little basic things like this without too much trouble. But the example blog post, of course, has the code where Albert is defining a custom JavaScript function that initializes an empty total of 0. And then basically for each row that's currently present using this state object, again, the state is like what JavaScript sees is what the user is seeing in that table.
So for example, if they're creating a filter, they're just gonna see what that state object is just gonna represent, what is actually showing on that table. Great. So he takes that and then for each of the rows that are in that state of the filter, he's gonna add up the values based on the column ID that's being supplied here and then return that total. So he does that for both the sales and the revenue along with some formatting of the revenue using some JavaScript, you know, functions as well, which, again, you can see in the show note I mean, in the blog post. And there you go. Now when you do that filtering, I'll even type it while I'm recording here.
Sure enough, when I filter for January, that total is now updating accordingly. So this is terrific. A terrific way to make sure that you get that shiny like experience, but again, this is all done in client side, and reactable has JavaScript bindings that you can customize as much or as little as you like. Lastly, speaking of making things look good, Albert wants to change the style of those, what I call, nested group rows versus the, you know, the rows of the actual data itself. His first attempt at this actually ends up coloring every row in the same color, which kind of defeats the purpose, doesn't it? But that's a artifact of using this row group this theme function with the reactable theme, function, and that, by proxy, is selecting every row for that theme. So, uh-oh, that's not quite good.
What can we do differently? Well, again, we can go in the JavaScript world where he defines a JavaScript function to say, you know what? I'm gonna take the row itself, which is called row info. It's like metadata for the rows each in each of the in each of the table. And for each of these groups, that first row, which is just showing the quarter name and whatnot, that's level 0. Remember, JavaScript index start at 0, not 1 like an r, so that might trip you up a bit. But he's saying if that level is 0 that's being shown, then do the custom background on the cell, you know, color.
Otherwise, leave the rest the heck alone. And sure enough, at the end product, you've got those quarter rows with that nice little, you know, shading so you can quickly visually distinguish those from the actual data cells as well. And last but not least, he does a little change up to the style of the footer, and this is again very simple to do. Again, a little bit of, CSS demonstrated here where you can add a footer style with custom CSS function from the HTML tools package, or you can give it, like, the font weight, and in this case a little line at the top called border top. So you can quickly see just like you might be writing this on paper, putting a line under the numbers you're calculating, and then making sure that visually pops out.
So again, a tour de force of how you can take a native reactable table that just comes with that data frame being fed into it, all the way to a very nice, polished, and again highly interactive, multiple levels, table for you to look at. Again, reactable and GT, wonderful packages for creating tables. Albert's been doing a lot of content on each of these, so you wanna check out the archive of his post for of his blog, I should say, for many of our posts that talk about using these these great packages in action. So I am thrilled thrilled to see this.
And rounding out our highlights today is another great showcase of another awesome piece of technology using a framework that's quite familiar to those that have been in the R Markdown ecosystem to save you a bunch of time and a lot of future effort. And, yeah, I think future you will thank you for investing in such techniques. We are specifically speaking about the Quarto publication system, which has been a hot topic ever since it was announced at Posiconf back earlier in 2022. I can't believe it's been that long, but, yeah, Quartle has been around a little bit already. And in particular, it has a lot of functionalities under the hood, but one of those which is definitely been inspired by the R Markdown predecessor is the idea of creating parameterized reports, in that you can compile a report and feed into it a set of values, I e parameters, to maybe update or dynamically change some content depending on those parameters.
And one of the real innovative thought leaders in this space of taking advantage of parameterized reporting from the Rmarkdown ecosystem and now Quartal is the author of this latest blog post on deposit blog, JD Ryan, who has had quite a career transition, I must say, as she has gone from being literally in the field or you might say in the streams working in the Washington State Department of Agriculture as an environmental technician, and now she is leveraging her data science skills more in the, I guess, an office type environment. But I had the great pleasure of seeing JD again at the aforementioned positconf earlier this month, and I must say for the many conversations we had, whether it was at the data science hangout or, you know, here pepping me up for my talk or even afterwards as the conference is winding down.
Yeah. Her, her energy is infectious and, sometimes I wonder where did I get that energy? Where can I get that? That fountain of youth that she seems to be drinking from? But nonetheless, it was great to touch base with her and and yes, she was kind enough to give me a little sticker too. And by the way, if you want some awesome swag, if you're a cats lover like she and I are and you want to flaunt R at the same time, you want to check out her awesome little shop on Etsy, which I'll link to in the show notes. Nonetheless, JD has been in this, world of creating privatized reports for quite some time, and she mentions how much it's been extremely helpful as she was transitioning to her data science role in terms of creating in upwards of hundreds of these reports for farmers across the area based on their particular data needs.
And instead of hard coding all this 1 by 1 for, like I said, over 300 of these, she was able to leverage the parameter equals the parameter functionality to create both interactive and static or PDF versions of these reports and have them dynamically render in a batch setting. So how do you actually get there? That's where the meat of the post dives in here, where there are different kind of high level goals that you want to look at depending on how far you want to take it. Obviously, if you only have a few reports, maybe maybe manual is the better ever for you, and that might be might be what you're used to. But then, as like I said, the scale starts to escalate.
If you find yourself regenerating a lot of these things manually, that's when you're on and start investing into the idea of parameterized reports. So how do they actually work? Well, no matter if you're doing a report based in r or based in Python, you will have a capacity to be able to specify these parameters either in the front matter YAML of that quartile report if you're leveraging the knitter engine, I. E. Using r, or if you're in the Jupyter setting, if you're leveraging Python, you can do a specific code chunk with a optional, tag attribute called parameters, and you can literally just type in the values 1 by 1.
Obviously, the parameter values are more optimized for, you know, textual type parameters, maybe numbers. You're probably not going to be able to use complex classes because it's got to be something that can be specified in text, by the renderer. But once you have that going and once you have those parameters defined, then you have many different ways to actually compile this. First of which is within a IDE like Rstudio or leveraging the quartile extension in Versus Code or positron, you can compile the report and it will use the default values of those parameters that you specified so you can get a feel for if the parameters are working as expected.
And so another option is maybe you want to keep those default values as is, but you wanna experiment with changing maybe 1 or 2 of them. Well, quartile is not like a r package, like rmarkdown was. It is actually a utility, a software utility on its own with its own command line interface. So you can run the quartal render function and feed in the quartal markdown document, you might call it template, and then you can specify 1 or more parameter values with the dash capital p flag, and then give the name of the parameter, a colon, and then that parameter value. So that's a great way if you don't want to change your your default parameters and you want to get a quick take on maybe changing those parameters on the fly, you can quickly do that.
Or if you don't want to type those command line flags, you could have a separate YAML file called params. Yaml, where then you can simply put them much like you would in that frontliner. Yaml for the default parameter settings, and just in the Kortal renderer call use the dash dash execute dash params flag, feed in that YAML, and now you've got again that that report compiled using those new values. And there is more, of course. If you are in the R ecosystem, there is the quartal R package, which again is not the quartal engine itself, it's just you can think of it as like an API type package to the quartal static rendering engine, where then that has an execute underscore params argument, and you can feed the parameters as a list object, and then feeding it as input the name of the file that you're going to render.
So you could do that from R itself, and now the wheels are probably turning here. You got all these different ways of executing and compiling the report with parameters. What are ways you can actually automate that? Well, there are a few techniques here and I've used a couple of these already, but they're they're really, really solid. One of which is to have a data frame which has all the combinations of the parameters that you're interested in rendering as each row. So in her example, she's got a year, a producer ID, and literally does a expand grid of these 2 year values and 4 ID values, and you'll get then this, 8 row data frame with each of those parameter combinations.
That's pretty straightforward, and what you do with that is you can augment then the different output formats you want. Maybe it's HTML, and then the actual file name using a little paste magic, and then you can have then a nested column that has these parameter values as lists, I. E. The columns that are already in this data frame, and make it a named list. And she's got a nice little snippet of code that does this with the purrr package, another one I simply can't live without, and again, if you love your cats, then how can you not love the purr package? So it seems like fit for purpose here. Then once you have that, then you can have a customized tidy kind of data frame organized by 1 row per iteration, and then back to purr you can use the handy function called p walk which you can feed in the data frame itself and then that basically means that each column of the data frame is like a function parameter That can get fed into the quartal render function, and those parameters are going to be fed very clearly, as named arguments.
So again this may take a little getting used to, but again you can leverage her snippets of code that she has in the post and try it out for yourself, and this can be part of a larger scale pipeline. Just imagine that you want to do these on a routine basis, you know, you can find ways to automate this. Heck, you you know, quarto is a first class citizen output format of the targets package that I speak so highly about. You could lever something like this in your targets reports as well. There are lots of interesting ways you can use this, technique, again, to save you a boatload of time and effort in the future.
So JD definitely has a lot more materials than what this blog post has here, and she's done a workshop about this in the past. She's done a presentation at Pazitconf, last year about this workflow and how it applied to her daily work. So, yeah, you'll be invited to check those out as well And again, love love the cat pics in this. I got it. Yeah. Makes me miss my cats from the yesteryear. But nonetheless, really great blog post by JD here. You can tell it's got all the bells and whistles of quarto itself in the blog post because it got nice little tabbed interface to go from the r snippet for that automated execution to a bash scripting, which links to another blog post by Solomon Moon who talks about how you can do this in a bash script. So again, whatever your flavor is, you've she's got you covered with different examples here.
And if that wasn't enough, boy, this issue has got a whole bunch in here that we wish we could cover today, and on in its entirety, but, yeah, time's not unlimited, unfortunately. But you can find all the additional resources at roku.org, and we'll take a minute here to talk about one of my additional finds that I think, might be worth looking at as you listen to the show and afterwards. And we heard announced at Posicomp in one of the talks a wonderful new extension for Quartle called Close Read. And what this is giving you is a way to have that kind of scrolly telling functionality in your quarter report that you often see in these websites that are kind of this hybrid of infographics and other, you know, neat neat utilities.
And the way you scroll through it, it kinda updates the content dynamically based on where you're at. It maybe freezes certain elements. Well, this this extension, Close Read, is really really great in this space. We haven't really had any we've had attempts at this before in the art community, but I think this one really nails it. So this, link is from Georgios Cameronis. I believe he also gave a talk at Posikov and this is a visual journey through world exhibitions and you kinda have to see it to believe it. I can't do enough, you know, enough justice on the audio form, but it's got this great interactive map that depending on where you're going in the story, it's gonna zoom in on different regions wherein it talks about the narrative behind these different exhibits.
It'll splice in some authentic pictures from these exhibits, but it's a great showcase of what is possible with close read and I am definitely going to take a look at this as I think about content that can be engaging to different audiences. I mean, gosh, my wish is I could do something like this in life sciences. I'm not sure if that type of material would be fit for it. Hey, you never know. Maybe so. But maybe I could find some other uses for it. But, Close Read is still early days, but it looks like a few people are already putting it through the paces. In fact, I saw another contributor to our weekly, highlights in the past, Nicole Raney has also done an example of her tidy Tuesday visualization in a close read document as well. We'll have a link to that in the show notes. So starting to see it in the wild, and I'm definitely interested in seeing where the uses of that extension end up going.
And yeah, like I said, there's a lot more to this issue, but we're gonna have to wrap things up here as as we're winding down. But of course, I like to always close with telling you how you can get involved to help the project, as well as this very podcast. First, where can you get involved with our weekly? Well, again, visit our weekly dotorg, and we welcome all your contributions to that great new blog post showcasing R and Data Science, maybe a great new package that you discovered or something that's had a new release, a great workshop tutorial, or maybe another podcast or videos out there.
Either way, there are different categories for each of these type of content, and the best way to do that is via a pull request already linked in the top right corner in that handy little ribbon on the Our Weekly site. You click that, you'll be taken to a predefined template. You'll fill out a little bit of metadata, and all we need is a link, all marked down all the time. Just like when you're writing that fancy quartile parameterized report, you use a markdown for that too. So we invite you to share your resources on there and the curator of the week will be glad to merge that in. And also, we are definitely open for a curator spots as well. We would love to hear from you on that front.
We have details on the process at rweekly.org, and that will take you to the GitHub repo where you have a nice reading that's been put together about how to get involved on the curation front. And also we'd love to hear from you on this very podcast, so we have a little contact page directly in the show notes that you can you can use to quickly send myself and Mike a message. You can also, if you're on a modern podcast app, some of my favorites are Podverse and Fountain. Also for iOS users, Casa Mac, I hear great things about. With one of those apps, you can send us a fun little boost along the way that goes directly to us on the on the podcast team and no middle party involved.
So that's available as well, but also you can find me on these social medias out there. Mostly on Mastodon these days at our podcast at podcast index.social. And I did, have a good friend of mine on one of the other communities I'm part of trying to figure out the best way to get started with Mastodon. If that is something you as well are trying to get into, I'll put a link to a couple, great write ups that I found helpful in the community. If you're new to your Mastodon journey, because I'm starting to see a lot of my friends from data science and the art community are starting to get on Mastodon, but the key is finding the best way to get in touch with them so you can find their content. So I have some resources about that in the show notes of this episode.
You can also find me on LinkedIn as well. Just search for my name and you'll find me on there and somewhat sporadically on the weapon x thingy with at the r cast. Well, that'll do it for this episode of Rookery Highways. Can't believe we're at episode 176 already. Gosh. We're 200 is not that far away, so hopefully we'll get there. One way or another, I hope. But again, next week I hope to have my trusty companion here, Mike, back on the mic with me, so that will be, you know, a welcome change instead of just hearing me blab all by myself. In any event, we're gonna close-up shop here that will wrap up episode 176 of our weekly highlights and we'll be back with a new episode next week.
