The ducks have made their presence felt in the world of databases, but now you can take a dip in a new Ducklake all within R! If you ever felt someone should write a book on the many ways you can slidecraft Quarto presentations, we have good news for you. Lastly, a spectacular series on harnessing the power of LLMs in your Shiny apps concludes with a powerful and very relevant use case to help review your next presentation slides.
Plus … a popular segment returns 😅
Episode Links
Plus … a popular segment returns 😅
Episode Links
- This week's curator: Eric Nantz: @[email protected] (Mastodon) & @rpodcast.bsky.social (BlueSky) & @theRcast (X/Twitter)
- Jumping in the Ducklake with nothing but R on
- Slidecrafting - Making beautiful slides with reveal.js and Quarto
- The Shiny side of LLMs Part 3
- Entire issue available at rweekly.org/2025-W40
- Use the contact page at https://serve.podhome.fm/custompage/r-weekly-highlights/contact to send us your feedback
- R-Weekly Highlights on the Podcastindex.org - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby, and then head over to the R-Weekly Highlights podcast entry on the index.
- A new way to think about value: https://value4value.info
- Get in touch with us on social media
- Eric Nantz: @[email protected] (Mastodon), @rpodcast.bsky.social (BlueSky) and @theRcast (X/Twitter)
- Mike Thomas: @[email protected] (Mastodon), @mike-thomas.bsky.social (BlueSky), and @mike_ketchbrook (X/Twitter)
- Relm's Weepy Loop - Final Fantasy 6 - MkVaff - https://ocremix.org/remix/OCR00178
- Serenity & Grace - Chrono Cross - timaeus222 - https://ocremix.org/remix/OCR04143
[00:00:03]
Eric Nantz:
Hello, friends. We are back with episode 212 of the R Weekly Holidays podcast. Boy, that number keeps going up and up, and sometimes it surprises even me. But I'm happy to have you join us wherever you are around the world. And if you're new to the show, this is the weekly podcast, usually weekly podcast, where we talk about the excellent highlights and other resources that are shared on this week's our weekly issue at ourweekly.0rg. My name is Eric Nance, and like I said, I'm happy to have you join us. It's already been a hectic morning for me personally where I was part of a R pharma diversity alliance hackathon with version control, and I'm happy to say no fires or get had to be put out. And we got three poll requests in. So I'm gonna call that a win. So already a a nice day going so far, but day's getting even better because who's joining me here on the virtual screen here? It is my awesome cohost, Mike Thomas.
Mike, do you have any git fires today? I hope everything's going well over there.
[00:01:01] Mike Thomas:
No git fires. Today's a good day because I am writing code. I'm doing targets quarto types intersection, and everything is working swimmingly for now.
[00:01:12] Eric Nantz:
I feel like I should say, oh my, afterwards, but that is a very gnarly combination. But, yeah, I'm really interested to see the fruits of those labors, and I've been knee deep in many workflow actions myself, whoever in targets or even in GitHub actions proper, and debugging those is always an adventure. But that's not the focus today. We got our weekly to talk about today. As always, let's check our notes here. Who was the the individual that curated this? Oh, it's a goof me. Okay. It was my turn in the rotation. So it's about every, what, eight weeks or so, I'm called upon again. So as usual, this is never an effort I do alone. We have, terrific help from our fellow contributors, your curators, I should say, as well as contributors like all of you around the world with your terrific poll requests, other suggestions, and even helping the site itself. So special shout out to good friend of mine from the shiny community, Dean Atelli, who I had a great chance to meet again at POSITCOMF.
He sent a PR to make our site looking even better, especially when you look at the all, tab of the r weekly and you get all those issues. I've loaded up once some nice CSS formatting there as always. Anything to improve the site, we are more than happy to get in there. But let's improve your our knowledge, shall we? So let's dive into our highlights today, and we lead off with one of the hot topics that definitely Mike and I saw at Pazacampa this year and the year prior. Man, there is a ton of momentum around duck DB folks. DuckDV is becoming, you might call the new SQL lite on steroids. It is just doing so much in the community for in memory data management, and it also has some awesome integrations with WebAssembly, other great things that we heard even at the conf with integrations with Shiny, so many integrations, even semantic searching. There's I could go on and on.
But there is a new part of DuckDV that definitely flew under the radar for me, But, this first highlight is talking about a new capability called Duck Lake. And what does this mean to an R user? So let's find out, shall we? This highlight has been authored by Novica Natkoff, who is, I believe, a data scientist at the Norwegian veterinary Institute, and he calls this post jumping into the duck lake with nothing but our on. So this has been, I guess, a newer capability offered by the the company behind DuckDV. You I guess the way to describe this, and I'm very new to this, kind of this hybrid of what you hear in what's called a data lake where you often have a huge collection of data, but also some metadata associated with it to help with the storage and retrieval of various records.
Well, DuckDB is getting in that town so to speak with Duck Lake bringing in these two concepts together. Now how do you actually get started with this? Well, that's where in the R community we've had the duck DB package available to us for, gosh, probably almost two years now I'd guess, somewhere around those lines. But within the package itself of course you get duck DB ready for you. But one of the selling points of duck DB is the extension system. There are extensions for all sorts of things. I remember playing with the extension for object storage retrieval and and and sending of of parquet files and whatnot. That's played nicely. Well, there's a new extension called the Duck Lake extension, and that's where once you load up DuckDV in your R process and you create a database, usually an in memory database to play with things, you can execute via DB execute similar to the DBI syntax the call to install literally the Duck Lake extension from the official DuckDV extension repository.
With that, you can now attach a new database. They call this ducklake dot metadata dot ducklake as a as a new in memory database, and this will create now this duck lake version on your disk. Certainly in production usage, this Duck Lake artifact would live on probably object storage of our cloud provider and whatnot. But with that, you can now play with this in your inyourrcom. So again, it's written locally to disk on a temporary basis to start with. So what's why do you need to care about this? So in the example in the blog post, Novica talks about let's load the m t cars set. You can load that in with a typical DB write table call, and then you can now create additional tables with this, say with a different color or a different categorization.
Where in this case, he's looking at a a derivation of the miles per gallon over the mean of the miles per gallon as a new as a new variable and then creating a summary of that with some averaging and then grouping that by the cylinders. So you've got now a layer of raw data. He's calling that bronze in this case, but this second layer that I just talked about is sort of some cleaning and normalization typical of any data kind of pipeline. And now if you look inspect that that new file based database duckleg that you've ex that you created here, you've got three files associated with these different layers of the table that he has created here. And in each of each of that you've got a parquet file. So it's using parquet as a storage format.
Again, to me, parquet is like the new CSV. It is just everywhere now. You can't escape it. Not that you want to escape it. It makes your life a heck of a lot easier, I would say, once you get to know that better. So now those are three kind of separate tables based on these derivations, but you want to figure out then something that I think is still on the bleeding edge here is being able to track kinda how these tables are related to each other. He does have some code that, that would be in the ideal case of how you define, like, parent tables, child tables when it's created over metadata.
I believe that's not quite working as expected, but there are some additional metadata you can put in the table, you know, Duck Lake, I should say, nonetheless, such as executing different values into the filtering operation and then visualizing then the relationship amongst these tables. So So you got these parquet files that are kind of stored on disk. Right? And that's where you can now select into these from these relationships that you've established. And, again, some code works. I believe some doesn't. Haven't tried it myself. But then you can create then a graph of the kind of ontology of the the way these tables relate to each other. And there's, there's an example here in the blog post. It looks like a kind of a DAG of sorts where it goes from one part of the table and I have to zoom in on my browser because it is kind of small on my screen here.
From the the the first level, he called it the bronze version of MT cars, and then it branches off into separate branches of these different derivations that have been made here with this silver layer. And then from one of those layers that branches off again to this gold layer at the very end, calling it kind of a a catalog of sorts when visualizing how those are related. So what he thinks the value of this is is that when you have these this metadata establishing the relationship amongst these tables, you can now put this relationship diagram of sorts, whether it's like this graph here in in the post or elsewhere, that could go in, like, your internal documentation for this, you know, set of databases or set of tables.
Thinking of Quarto that Michael Mike mentioned he's been working on. Maybe a Shiny app that visualizes the relationships amongst these. There are a lot of things that this could be used for, and in fact, he even puts that at the end on top of the gist that has the code for the blog post. Maybe you want to fold this into part of an existing data pipeline. We covered in, I think, last year's our weekly highlights a package called Maestro that helps you kinda take a workflow like approach to building data pipelines. This could be a good fit for those as well.
So my take on this is that, a, it's standing on on well established format for saving these the disk, a parquet, but it's not just the derivations themselves. You are able to add kind of relationships for these tables as metadata and then have that available however you choose to visualize it. So I'm still very new to this. I do want to play with this a bit once I get into more complex setup, but it's great to see that with an extension away, you can now play with a little bit at your fingertips of what the Duck Lake Promise sets to be. So really informative post and and shout out, Novica who sent this my way on blue sky, and I'm like, yes, please. Get me all the Duck DB content you can you can pass at me. So I'm enjoying this, but, yeah, Mike, does this get your creative juices doing what or going what you do with DuckDV?
[00:11:10] Mike Thomas:
It does very much so. And I'm loving DuckDV. It's still pigeonholed on a lot of projects with clients that we have on some of the larger, you know, data lake, if you will, like engines and providers or Databricks and your Snowflakes of the world. And I just anytime I'm working on one of those projects, I always feel like if I had the opportunity to start now from scratch, we wouldn't need it. We could just use DuckDV, Parquet files, and things like DuckLake here to essentially do most of the same stuff for free in a completely open source way that we would have more control over as well. And and I think the benefits of DuckDV are not something that we need to go into again because I've probably, talked about that way too much on this podcast. I just talked about that probably for five or ten minutes at Positconf, last week, really, how great of a project that is. And there's a lot more coming out on that project as well. Duck Lake being one of them. I think Mother Duck as well as sort of a tangential project that allows you to leverage, you know, cloud compute, and distributed computing, I believe, as opposed to needing to run things on your own laptop if you want. So a lot of advancements, going on in this space. And I think one of the the real pillars of of what Duck Lake is able to do and what Novica is, demonstrating in the blog post is kind of this concept, and you may have heard it if you're familiar with things like Databricks or Snowflake, of of time travel, really. Such that you can start out with an initial dataset or a table, which is under the hood gonna be represented in a parquet file, stored in a parquet file. And any changes that you make to that, you know, through SQL update statements, through SQL, insert or delete statements will also be tracked in subsequent parquet files within that same storage.
And there's this abstraction layer that you can write SQL sort of on top of that storage, and the way that the files are named and organized in this, Duck Lake convention, which is is not necessarily new as you mentioned. This this data lake protocol has been around kind of for a while. But the, sort of the protocol allows you to write SQL on top of it to say, you know, query, this table as of this particular time or as of this particular version. And under the hood, it can figure out what, data it needs to query, what inserts to apply, what deletions need to be applied as well to show you the data that you're essentially asking for. So it kind of builds up this set of parquet files in your storage.
And I I think this kind of time travel notion is the biggest sort of selling point of, what Duck Lake has to offer right now. And I really appreciated Novica taking that one step further in places where, you know, I'm not sure, DuckDV has, like, visual representations yet, but trying to actually show lineage as well beyond that. So I thought that that was really cool. And, you know, DuckDuck, I think, out of the box, tries to recommend leveraging Parquet for file storage and Postgres for essentially your data catalog, and allowing you to or your metadata, if you will, on top of your, data warehouse or your your data lake that you're creating. So that combination is is interesting as well, maybe good to know in terms of context here. But I I really enjoyed, this write up because I I haven't seen a whole lot of, you know, diving into some of the the weeds in these new developments in what's in DuckDV. I've seen a lot of getting started, but, in terms of getting started with, Duck Lake here, it's not something I had seen a whole lot of before. So I really appreciate Novoca, putting this together for us.
[00:15:04] Eric Nantz:
Yeah. I'm definitely gonna pursue this and maybe some lower lower risk projects just with some assorted simulated data, but I could I could see this time travel aspect. Not some ideal with a whole lot on, you know, strictly clinical data kind of stuff I do for most of the the day job. But certainly as we look at different evolving stages of, say, assumptions about data and different derivations across its life cycle, this this could be a great tool in our toolbox. So this is this is in the highlights for a reason. I think this is a a great way to to start off this episode. Well, Mike, you mentioned you had a a spectacular presentation that you had at Posikov, and and I again, I watched the recording again. It was just as I thought when it was live.
But you and I had something in common with with that other than just, of course, us being presenters at Pazitconf. We really like using Quarto. We've revealed JS to create our slides these days. It is just been an absolute pleasure compared to what I usually have to do with the day job, which I will save that rant for another time. You probably know what I'm talking about. But it is really, really handy to take what I'm comfortable with, mark down syntax, and break these up into literally different sections of my slides. If I need to style it up, I sure can. You know, if I don't need to style it up and I just want nice convenient things like two column layouts, embedding code chunks, annotating through those code chunks, highlighting things as I progressively build, all that comes out of the box of reveal JS and quarto. Absolutely love it.
And I have bookmarked a lot of stuff in my collection of bookmarks with the quarto tag, and they're either from people that made quartal or those that are like us making slides of quartal and they do some neat little trick and I'm like I gotta bookmark that. There's a lot of these tips out there. A lot of them are quite scattered, but there's been one person at the forefront of collecting these in such a way that we could benefit as a community. And boy, he has taken that even up a 100 notches here. So we are speaking of the newly released online book called slide crafting, authored by who has been one of the ones at the forefront of this.
Emil Fithfeld, who is on the as a posit engineer at Posit does a lot of work with the the tiny models team, but he has also been making so many great resources about Quarto and in particular making slides of Quarto. His blog post that he called slide crafting had been featured as previous highlights before but now he has assembled them into one place called the slide crafting book and this is still early we always saw this come out I believe about a week ago or so it was after comp for sure and it's got a collection of all these tips that have been scattered or he has mentioned in his previous blog post but it's of it's not feature complete yet but what it has here is of definitely enough to get you started on the right foot.
All with kind of a crash course of creating your own custom theme for your coral slides in ten minutes or less. It literally has from start to finish what that process looks like, and then the rest of the book goes a deeper dive into each of these. What goes into the theme? Right? What colors do you select? What kind of fonts are you using? And how big or small are certain elements? Also, like I mentioned, what is the CSS styling associated with it? All of these are dedicated sections in this book, and I've already, just looking at this, have learned so much already that I'm gonna take into my next portal presentation.
And that's just the theming side of it. There's also the content block where we're gonna look at different ways that you can display your code, different ways you can highlight certain elements in your deck, changing layout. Where do things go? Left, right, moving it up and down? All these things that you can do with Quarto are all documented here. Like I said, it's not quite done yet. It's even got, other sections such as when you're showing a code demo using things like ASCII cast to help with that kind of recording like setup of a code chunk, ways you can bring interactivity via fragments and highlighting certain blocks of of of of elements, changing the styling on the fly, and even the the start of an extensions, section which is still not there yet, but I am looking at I'm gonna keep watching this quite closely.
There's a lot here. I've always wanted something like this because I want something where I can go to just in one bookmark, get all these tips. At some point, a lot of it will become muscle memory. It hasn't yet. So every presentation, I've learned something new. I do something different. In the past one at POSITCONF, I did the the highlighting of the code and narrated through each of those sections of, like, a shiny state block, but there's just so much. There's so much to learn. I think this is the start of something huge here. So, yeah, Is the slide crafting? Yes, it is, and I am here for it. What do you think, Mike?
[00:20:58] Mike Thomas:
Yeah. I I can't agree more. I am slightly upset that this came out how many weeks after Positconf?
[00:21:05] Eric Nantz:
I think maybe a week after.
[00:21:08] Mike Thomas:
Oh, man. I could have used, like, so much of this in my presentation. This is this is fantastic. And it's so many it answers so many of the little questions that I've had and just kind of not done the deep research into to figure it out, like image placement, layouts, things like that that are just, these great quality of life improvements for your presentation that I knew I needed to incorporate, but just didn't have the the bandwidth or didn't necessarily have a resource like this that I could go to to quickly get answers to my questions. And don't get me started on what the LLMs fail to know about, Quarto, reveal JS at this point because I I tried and and struggled quite a bit with some of the responses I was getting. But, anyways, this is it's just gonna be this is almost like a big book of type of a resource, and and I it's not super dense either. I mean, there's a ton of content here, but it's pretty easy and well organized to get to what you might be looking for. And even if you're not looking for something specific, you know, like I did, in preparation for the show today, I found myself, you know, scrolling through every section, you know, taking two minutes per section and and picking up something, constantly in terms of, theming and SCSS, which is, I promise, not as scary as it it may sound if you're not familiar with CSS or or SCSS.
It's maybe a little different to look at at first, but the amount of control that it gives you to incorporate your, you know, companies, or your organization's brand and logos and colors into your slide presentation that that that go a long way, believe it or not, to getting folks to pay attention to what you're presenting to them. That all everything is essentially right here. And some fantastic little things that I could have certainly used. I had, some slides in my presentation that I needed to kind of manually we had a clicker that we were holding, but I needed someone may out there may have the same type of situation. But some had elements in the slide that I needed to scroll down, and your clicker can't necessarily do that. You would have to go to your mouse or your touchpad on your laptop to be able to do that. It's not just advancing a slider, not advancing a slide. Well, of course, ML has a section in here, I believe, under, fragments potentially that shows me exactly how I could have done that.
You know, advance a tab set or change tab sets, I should say, or scroll, on some some content that's on your slide through, you know, the back and forward, buttons in the presentation. Oh, man. I could've used that so much. But, anyways, just a fantastic resource. I can't even pick sort of one area to highlight because I was just picking up stuff everywhere throughout this book. So I'm excited to see how it continues to expand, but a ton of content here, a great start. I think it's easily digestible. And if you go to this URL and don't find something, that, you know, you learned that you didn't know before, I'll give you a dollar. It's how confident I am. This is amazing here. And one thing I'm
[00:24:35] Eric Nantz:
as I'm assembling my notes through various presentations, even if I see a potential to hopefully add maybe to this, I'm hopeful that this will be a community driven effort as well. I may have some things to add around the video, playing when you get embed videos in coral. There's some neat tricks you can do with that as well, which were really handy for my talk with the the the Mega Man inspired app demos that went without a hitch. Thank goodness for that because I was a late night hack to get there, but that that that would be another time. But, yeah, even you, Mike, you utilize videos nicely in your part of the presentation and do a poll request and and issue notes. So, yeah, I could see. I yeah. This is definitely growing, but, boy, there is enough here now that for your next quartal presentation, whenever you're doing this at the day job or you have an external conference you're presenting at, I think you're gonna have enough here to do a a terrific job from both the UX side of it and, like I hope, have fun doing it because it is a lot of fun to make these.
Well, if I'm the curator for our weekly issue, you know I'm gonna have something with Shiny involved in it and, and, and, podcast. So, luckily, this was voted by others as well, not just me. But especially at Pazitconf and many avenues before this, we are seeing very much the intersection of large language models to enhance both development but also giving unique experiences to Shiny apps that just simply were not possible even just a couple years ago. There has been a terrific series that kinda takes you from literal start to finish, so to speak, on how you might approach building LLM AI model capabilities in your next Shiny app. And so we are talking about the third in this series called the shiny side of LLMs, and this has been authored by Vera Elfink van Leeenpoot who has been a prolific member in the shiny community since near the beginning.
She has done tremendous resources. She's also doing courses about Shiny as well. We have a platform called Athletics that I've heard great things about, but in any event she has been embarking on this journey to bring up to speed somebody who's new to the world of language models, what are the nuts and bolts behind how it works, but now we're at the third part where we actually plug this in to a shiny app. So the first parts are talking about what elements actually do and what they don't And then the next part talked about going from prompts that you put into the LMS interfaces and how the responses are constructed.
Like I said, now it's time to put this in action. And this comes full circle when we last talked about because the use case for this is going to build a Shiny app that will let you upload the contents of your quarto presentation and get feedback from the LOM on what's working well and what might be tweaked. So it is in essence a peer reviewer of your next portal slide presentation. Oh, boy. You told all things I wish I had had sooner. Maybe I could have used this before the conference talk, but nonetheless, better late than never.
So she's calling this app deck check, and it's gonna start with some narrative here and then the post about first why we choose Shiny. Well, I think the question is why not, but that's another point for another day. The point being is that within the ecosystem, now we have packages both on the our side and the Python side to integrate these chat like interfaces into your app this is called shiny chat on your side and chat last on the Python side so once you have that ready to go and and install either of those packages for your use case, you've now got to choose the LOM provider, of course.
I've been a big fan of Anthropics Cloud Code. Their models have done really well for me, but, nonetheless, whichever either hosted platform you choose, you're likely gonna have to get an API key because this is gonna go into your configuration based with the Elmer package to make all this work. But like I said there are code examples here to build this app literally from beginning to end and we start off with getting the chat interface going outside of shiny so I'm gonna focus on the r tabs here in this what looks like a cordial blog post with Elmer and configuring the chat tie into anthropic so there's a chat underscore anthropic object that you create where you just say the model the system prompt that you give it, again, plain text, and any additional parameters specific to the model itself.
And then this is something new. It's not just the chat object, but you want to plug this in the Shiny. Right? And for this, you want to be able to kind of have that nice streaming effect as the response comes in, and that's where Elmer comes with the stream method of this chat object. And then that's where you feed into it some language that you want displayed in the app itself, and then be able to tell the LOM what to do with that. I've never played with the stream object before, so I'm gonna or stream methods. I definitely want to play with this after after this, after this episode probably.
It does some unique generator based on the coral package. I believe that's a way to kind of do the looping correctly, but this is this is really neat. So it gives you that natural look and feel when the LOM is responding back after the user uploads their coral presentation. The other thing to keep in mind is that you don't want to have a bottleneck in your app as this response comes in. You want to take advantage of what's called asynchronous processing. This is great for the UX of all this so that your chat doesn't grind things to a halt with the rest of the app as the response comes in that's where it's a combination of again the Elmer package and in this case the promises package on the our side to do the special chat underscore async function.
So this is flagged to do asynchronous mode for this and it returns what's called a promise object afterwards that will be rendered or resolved when the app needs it, but it's not gonna stop everything else. So with that, you've got kind of the building blocks in place, and now it's time to build the UI. This is where, for preparing my talk at the conf, I did a, vibe coding session to prepare the UI in my demo and I got like 90% of the way there. But here she's done the hard work for you. A simple layout would b s lib, got a nice of sidebar or page fluid layout, I should say, where we've got a module for the chat component and a header at the top kinda at, like, a little placeholder for the user to ask anything about the presentation.
Very simple UI. Again, a module chat mod UI is gonna drive most of this from the Shiny chat package. On the server side, it's making that connection to the anthropic model and then rendering the server side of this chat interface, chat mod server. They've done the hard work for you. This is a module plug and play. You don't have to reinvent the wheel here. Really great to see the progress on this because I saw the early days of it. It was a lot of custom code. Now, right off the shelf with ShinyChat. So once you have that, then you've got the scaffolding of a mess of a new UI. It's very straightforward right now. We're just seeing, does this work?
And then you see in the screenshot here the prompt that was added as part of that metadata going into the chat object and then the response from the LOM based on the title that the user entered they call it the shiny side of LOM so what's your feedback and it's got the response there so we know the the plumbing works right now it's time to make the app look nice we can't just do that one column interface right so now she does a little whiteboard in here a sketching here of what the app will look like, a nice sidebar layout with cards that are added to the application. We got a main content and and then there's a nice animated screen of what they're working towards, the app in action.
The LOM prompt is on the left, the chat interface, there's a nice little CSS loader icon, and then once it's done it's got a whole bunch of like a series of value boxes at the top. Things like length of presentation and other content, and then the actual advice that it has for the the given presentation with various pieces of feedback. So all the code is here for both R and Python, and you get again this very nice UI and the app itself is doing a lot under the hood. Right? It's actually taking that uploaded quarto file and then it's running quarto under the hood to get the HTML version and then it's getting that that HTML file which is then fed to the l o m based on what they've added in in the back end of it. So this is a nice kind of diagram showing how this process works but in the end you can copy this app right from the blog post and try it out for yourself providing you've got that key to anthropic or whatever AI service you want to you want to do here. So there's other enhancements that she talks about here, such as a loading experience where you get that nice little loading icon when things are happening. So the user's not left with just a white screen doing nothing.
Those nice value boxes at the top, nice refinements. Again, just giving that extra nudge, that extra pizzazz to the user experience. But there's a lot more to this. There's styling. There's how you handle error checking. But this is as comprehensive of a post I've ever seen to take one of these apps from start to finish. So there's some nice nice examples, like I said, for error checking. And then when you're ready to go, there's a great section, what you would do to deploy this. Obviously, there are platforms like Pasa Connect that would be able to take this, Shiny apps IO, the internal I should say the internal Pasa Connect or the external Pasa Connect cloud.
But if I dare say so myself, another way you could host this would be containers as well and plug that into whatever app, you know, of service a, lets you host a container. Again, you build the app. You've got many choices where you host it. Either way, as always, if you're gonna put something like this for public facing, wherever internal your organization or in the public domain, be wary of the usage of that API key because you don't wanna break the bank too. But in the end, for a small app that doesn't have more than a handful of users, I doubt you'll you'll do very much.
Add Jason to this post. I did I had a colleague ask me, hey. You've been using Anthropic for a while. What's been your cost of this? Looked back at my cost. I kid you not, Mike. From January to now, my cost for anthropic has been a whopping $3.20. It's time you did some explaining. That's less than a lunch at the cafeteria. So I dare say I'm doing things right, I hope.
[00:37:00] Mike Thomas:
I don't believe I believe it.
[00:37:02] Eric Nantz:
Alright. But yeah. Yeah. That's so that's where just from little me developing and using it with Shiny or Positron assistant and in some very low level apps that that's pretty tangible. But, again, fantastic series of posts here. This is a great, great example of taking the concepts from the first parts. Now put it in action for a literal use case that I most definitely will be using before my next presentation. But, Mike, you've been we've been watching this space for a while. What did you think of Verway's exploration
[00:37:35] Mike Thomas:
here? It would be one thing for Vira to just, you know, use some sort of toy example and go through the entire process of creating both R and Python Shiny apps with the LLM capabilities in the back end, the ability to render the Quarto document. Doing that all in just one language would be incredible. She's done it in both languages and with a use case that is incredibly, I don't know, like, relevant or or tangible, practical, for anybody that's having to make a quarto presentation, which, you know, we just talked about in the last blog post and we've both done recently. And by the way, your presentation at Pazitconf was was fantastic. I think you gave me a little too many flowers there, compared to your presentation. But, this is, as you mentioned, probably one of the most comprehensive blog posts I've ever seen. I think it's well deserved that it's on, Pozits, series of shiny blog posts. And a I'll just pick out maybe a couple things that I I really like.
I think the error handling is something that we talk about a lot, Eric, and ensuring that your users aren't suddenly just seeing, like, red letters and error messages that are cryptic on their screen, and you do your best to try to this is an approach we always take is maybe pop up a modal when there is an error that tells them, hey. Something went wrong. Maybe you're able to to provide some pass or if statements that categorize what went wrong to give them a message, or maybe you even just voice the the message itself, you know, that you would see in the the terminal in that modal, but it's a a much better user experience than the whole app essentially just breaking. Right? We certainly don't want that to happen, if possible.
The the UI and the design here, I appreciated the time that she took to wireframe, what she wanted to put together, before that. And it's also you know, I don't think this is the purpose of the blog post, but even just some of the ways that she went about crafting her her prompt, yeah, a system prompt, if you will, to sort of instruct the LLM what it should do, you know, how it should respond and things like that were really interesting to me. She does have a system prompt in there, and then she has sort of a a more verbose regular prompt.
And the results look fantastic. I'm very excited. I have my cloud API key ready, and it seems like I can just sort of copy and paste this this code locally and try to run it myself. Myself. And I'm quite sure I'm gonna be blown away here, but a fantastic deep, deep dive into the art of the possible with Shiny, for R, Shiny for Python, and their associated LLM, counterparts these days.
[00:40:37] Eric Nantz:
Yeah. I am, literally, this afternoon, gonna build a custom next dev environment with Rex just for this because I am that eager to try it out. So, absolutely, that is on my on my list here. And, again, this was the third part, the culmination of a three part series. So definitely, like I said, if you're new to this, definitely check out her existing material. It does a great job of breaking down what can be very ambiguous terms if you're new to this. I mean, even me personally, I still get tripped up by some of the terminology thrown left to right on LinkedIn posts or whatever else in conversation. So I definitely like to have it explained to me practically, but not not in a condescending way. Verily does a terrific job with this. So, yeah, this is this is right in my wheelhouse, and, yeah, I'm doing still doing chordal slides. Might have to do one again before the end of the year, so this is this is gonna be utilized for for darn sure. But, we hope you also utilize the rest of the r weekly issue. We are running a bit strapped for time so no additional fines here. I do well, maybe selfishly speaking, I do have in this issue my adventures with switching the shiny dev series site to corridor so apropos of what we talked about today lots of interesting findings there and I'm back to blogging now. I got a blog component of shiny dev series so, who knows how long it'll last but I'm gonna try and be diligent with it, but it was it was it was good to get get the creative juices going again. So that's in there but also many many other things too. So with that we will, wrap up here but, of course, we love hearing from you in the community and we got many ways to get in touch with us. You can send a poll request.
That's time that's definitely appropriate after what I went through this morning, a poll request to the markdown files that comprise the our weekly project, especially the next issue. We have a draft markdown file. If you find a great resource, new package, great blog post, maybe you wrote a great blog post you want it featured, yeah, just send a poll request to that draft issue template and the next curator will be glad to merge it in. And, also, you can get in touch with us on the social medias out there. You can find me on Blue Sky these days. I am at r podcast dot b s k y dot social.
Also, I'm Mastodon at [email protected]. That's a mouthful sometimes. And I'm also on LinkedIn. You can search my name and you find me there. Mike, where can they get a hold of you? You can find me on
[00:43:10] Mike Thomas:
blue sky at Mike dash Thomas dot b s k y dot social, or you can find me on LinkedIn if you search catch Brook analytics, k e t c h b r o o k. You can see what I'm up to.
[00:43:23] Eric Nantz:
Awesome stuff. Then again, big shout out to all the listeners out there that said, hi, nose back at the conf. I will always appreciate all those kind words. We're trying to get back to the regular schedule here where we're doing our part, trying to anyway. But, nonetheless, we'll close-up shop for episode 100 212, I should say, of our weekly highlights, and we will be back with another episode of our weekly highlights next week. Hello, friends. We are back of episode 102 212.
[00:44:02] Mike Thomas:
Well, they so I had my speaker notes, you know, on one screen Right. And then my slides on the other screen. And I thought that if I
[00:44:10] Eric Nantz:
pressed play on the video on my speaker notes that it would translate into the video. They're separate. They're separate. I found out the hard way you know before. Yep. Oh, man. Okay. I didn't know he had videos in yours. I would have told you, but No, I know. I'll have to check yours out. You're good.
Hello, friends. We are back with episode 212 of the R Weekly Holidays podcast. Boy, that number keeps going up and up, and sometimes it surprises even me. But I'm happy to have you join us wherever you are around the world. And if you're new to the show, this is the weekly podcast, usually weekly podcast, where we talk about the excellent highlights and other resources that are shared on this week's our weekly issue at ourweekly.0rg. My name is Eric Nance, and like I said, I'm happy to have you join us. It's already been a hectic morning for me personally where I was part of a R pharma diversity alliance hackathon with version control, and I'm happy to say no fires or get had to be put out. And we got three poll requests in. So I'm gonna call that a win. So already a a nice day going so far, but day's getting even better because who's joining me here on the virtual screen here? It is my awesome cohost, Mike Thomas.
Mike, do you have any git fires today? I hope everything's going well over there.
[00:01:01] Mike Thomas:
No git fires. Today's a good day because I am writing code. I'm doing targets quarto types intersection, and everything is working swimmingly for now.
[00:01:12] Eric Nantz:
I feel like I should say, oh my, afterwards, but that is a very gnarly combination. But, yeah, I'm really interested to see the fruits of those labors, and I've been knee deep in many workflow actions myself, whoever in targets or even in GitHub actions proper, and debugging those is always an adventure. But that's not the focus today. We got our weekly to talk about today. As always, let's check our notes here. Who was the the individual that curated this? Oh, it's a goof me. Okay. It was my turn in the rotation. So it's about every, what, eight weeks or so, I'm called upon again. So as usual, this is never an effort I do alone. We have, terrific help from our fellow contributors, your curators, I should say, as well as contributors like all of you around the world with your terrific poll requests, other suggestions, and even helping the site itself. So special shout out to good friend of mine from the shiny community, Dean Atelli, who I had a great chance to meet again at POSITCOMF.
He sent a PR to make our site looking even better, especially when you look at the all, tab of the r weekly and you get all those issues. I've loaded up once some nice CSS formatting there as always. Anything to improve the site, we are more than happy to get in there. But let's improve your our knowledge, shall we? So let's dive into our highlights today, and we lead off with one of the hot topics that definitely Mike and I saw at Pazacampa this year and the year prior. Man, there is a ton of momentum around duck DB folks. DuckDV is becoming, you might call the new SQL lite on steroids. It is just doing so much in the community for in memory data management, and it also has some awesome integrations with WebAssembly, other great things that we heard even at the conf with integrations with Shiny, so many integrations, even semantic searching. There's I could go on and on.
But there is a new part of DuckDV that definitely flew under the radar for me, But, this first highlight is talking about a new capability called Duck Lake. And what does this mean to an R user? So let's find out, shall we? This highlight has been authored by Novica Natkoff, who is, I believe, a data scientist at the Norwegian veterinary Institute, and he calls this post jumping into the duck lake with nothing but our on. So this has been, I guess, a newer capability offered by the the company behind DuckDV. You I guess the way to describe this, and I'm very new to this, kind of this hybrid of what you hear in what's called a data lake where you often have a huge collection of data, but also some metadata associated with it to help with the storage and retrieval of various records.
Well, DuckDB is getting in that town so to speak with Duck Lake bringing in these two concepts together. Now how do you actually get started with this? Well, that's where in the R community we've had the duck DB package available to us for, gosh, probably almost two years now I'd guess, somewhere around those lines. But within the package itself of course you get duck DB ready for you. But one of the selling points of duck DB is the extension system. There are extensions for all sorts of things. I remember playing with the extension for object storage retrieval and and and sending of of parquet files and whatnot. That's played nicely. Well, there's a new extension called the Duck Lake extension, and that's where once you load up DuckDV in your R process and you create a database, usually an in memory database to play with things, you can execute via DB execute similar to the DBI syntax the call to install literally the Duck Lake extension from the official DuckDV extension repository.
With that, you can now attach a new database. They call this ducklake dot metadata dot ducklake as a as a new in memory database, and this will create now this duck lake version on your disk. Certainly in production usage, this Duck Lake artifact would live on probably object storage of our cloud provider and whatnot. But with that, you can now play with this in your inyourrcom. So again, it's written locally to disk on a temporary basis to start with. So what's why do you need to care about this? So in the example in the blog post, Novica talks about let's load the m t cars set. You can load that in with a typical DB write table call, and then you can now create additional tables with this, say with a different color or a different categorization.
Where in this case, he's looking at a a derivation of the miles per gallon over the mean of the miles per gallon as a new as a new variable and then creating a summary of that with some averaging and then grouping that by the cylinders. So you've got now a layer of raw data. He's calling that bronze in this case, but this second layer that I just talked about is sort of some cleaning and normalization typical of any data kind of pipeline. And now if you look inspect that that new file based database duckleg that you've ex that you created here, you've got three files associated with these different layers of the table that he has created here. And in each of each of that you've got a parquet file. So it's using parquet as a storage format.
Again, to me, parquet is like the new CSV. It is just everywhere now. You can't escape it. Not that you want to escape it. It makes your life a heck of a lot easier, I would say, once you get to know that better. So now those are three kind of separate tables based on these derivations, but you want to figure out then something that I think is still on the bleeding edge here is being able to track kinda how these tables are related to each other. He does have some code that, that would be in the ideal case of how you define, like, parent tables, child tables when it's created over metadata.
I believe that's not quite working as expected, but there are some additional metadata you can put in the table, you know, Duck Lake, I should say, nonetheless, such as executing different values into the filtering operation and then visualizing then the relationship amongst these tables. So So you got these parquet files that are kind of stored on disk. Right? And that's where you can now select into these from these relationships that you've established. And, again, some code works. I believe some doesn't. Haven't tried it myself. But then you can create then a graph of the kind of ontology of the the way these tables relate to each other. And there's, there's an example here in the blog post. It looks like a kind of a DAG of sorts where it goes from one part of the table and I have to zoom in on my browser because it is kind of small on my screen here.
From the the the first level, he called it the bronze version of MT cars, and then it branches off into separate branches of these different derivations that have been made here with this silver layer. And then from one of those layers that branches off again to this gold layer at the very end, calling it kind of a a catalog of sorts when visualizing how those are related. So what he thinks the value of this is is that when you have these this metadata establishing the relationship amongst these tables, you can now put this relationship diagram of sorts, whether it's like this graph here in in the post or elsewhere, that could go in, like, your internal documentation for this, you know, set of databases or set of tables.
Thinking of Quarto that Michael Mike mentioned he's been working on. Maybe a Shiny app that visualizes the relationships amongst these. There are a lot of things that this could be used for, and in fact, he even puts that at the end on top of the gist that has the code for the blog post. Maybe you want to fold this into part of an existing data pipeline. We covered in, I think, last year's our weekly highlights a package called Maestro that helps you kinda take a workflow like approach to building data pipelines. This could be a good fit for those as well.
So my take on this is that, a, it's standing on on well established format for saving these the disk, a parquet, but it's not just the derivations themselves. You are able to add kind of relationships for these tables as metadata and then have that available however you choose to visualize it. So I'm still very new to this. I do want to play with this a bit once I get into more complex setup, but it's great to see that with an extension away, you can now play with a little bit at your fingertips of what the Duck Lake Promise sets to be. So really informative post and and shout out, Novica who sent this my way on blue sky, and I'm like, yes, please. Get me all the Duck DB content you can you can pass at me. So I'm enjoying this, but, yeah, Mike, does this get your creative juices doing what or going what you do with DuckDV?
[00:11:10] Mike Thomas:
It does very much so. And I'm loving DuckDV. It's still pigeonholed on a lot of projects with clients that we have on some of the larger, you know, data lake, if you will, like engines and providers or Databricks and your Snowflakes of the world. And I just anytime I'm working on one of those projects, I always feel like if I had the opportunity to start now from scratch, we wouldn't need it. We could just use DuckDV, Parquet files, and things like DuckLake here to essentially do most of the same stuff for free in a completely open source way that we would have more control over as well. And and I think the benefits of DuckDV are not something that we need to go into again because I've probably, talked about that way too much on this podcast. I just talked about that probably for five or ten minutes at Positconf, last week, really, how great of a project that is. And there's a lot more coming out on that project as well. Duck Lake being one of them. I think Mother Duck as well as sort of a tangential project that allows you to leverage, you know, cloud compute, and distributed computing, I believe, as opposed to needing to run things on your own laptop if you want. So a lot of advancements, going on in this space. And I think one of the the real pillars of of what Duck Lake is able to do and what Novica is, demonstrating in the blog post is kind of this concept, and you may have heard it if you're familiar with things like Databricks or Snowflake, of of time travel, really. Such that you can start out with an initial dataset or a table, which is under the hood gonna be represented in a parquet file, stored in a parquet file. And any changes that you make to that, you know, through SQL update statements, through SQL, insert or delete statements will also be tracked in subsequent parquet files within that same storage.
And there's this abstraction layer that you can write SQL sort of on top of that storage, and the way that the files are named and organized in this, Duck Lake convention, which is is not necessarily new as you mentioned. This this data lake protocol has been around kind of for a while. But the, sort of the protocol allows you to write SQL on top of it to say, you know, query, this table as of this particular time or as of this particular version. And under the hood, it can figure out what, data it needs to query, what inserts to apply, what deletions need to be applied as well to show you the data that you're essentially asking for. So it kind of builds up this set of parquet files in your storage.
And I I think this kind of time travel notion is the biggest sort of selling point of, what Duck Lake has to offer right now. And I really appreciated Novica taking that one step further in places where, you know, I'm not sure, DuckDV has, like, visual representations yet, but trying to actually show lineage as well beyond that. So I thought that that was really cool. And, you know, DuckDuck, I think, out of the box, tries to recommend leveraging Parquet for file storage and Postgres for essentially your data catalog, and allowing you to or your metadata, if you will, on top of your, data warehouse or your your data lake that you're creating. So that combination is is interesting as well, maybe good to know in terms of context here. But I I really enjoyed, this write up because I I haven't seen a whole lot of, you know, diving into some of the the weeds in these new developments in what's in DuckDV. I've seen a lot of getting started, but, in terms of getting started with, Duck Lake here, it's not something I had seen a whole lot of before. So I really appreciate Novoca, putting this together for us.
[00:15:04] Eric Nantz:
Yeah. I'm definitely gonna pursue this and maybe some lower lower risk projects just with some assorted simulated data, but I could I could see this time travel aspect. Not some ideal with a whole lot on, you know, strictly clinical data kind of stuff I do for most of the the day job. But certainly as we look at different evolving stages of, say, assumptions about data and different derivations across its life cycle, this this could be a great tool in our toolbox. So this is this is in the highlights for a reason. I think this is a a great way to to start off this episode. Well, Mike, you mentioned you had a a spectacular presentation that you had at Posikov, and and I again, I watched the recording again. It was just as I thought when it was live.
But you and I had something in common with with that other than just, of course, us being presenters at Pazitconf. We really like using Quarto. We've revealed JS to create our slides these days. It is just been an absolute pleasure compared to what I usually have to do with the day job, which I will save that rant for another time. You probably know what I'm talking about. But it is really, really handy to take what I'm comfortable with, mark down syntax, and break these up into literally different sections of my slides. If I need to style it up, I sure can. You know, if I don't need to style it up and I just want nice convenient things like two column layouts, embedding code chunks, annotating through those code chunks, highlighting things as I progressively build, all that comes out of the box of reveal JS and quarto. Absolutely love it.
And I have bookmarked a lot of stuff in my collection of bookmarks with the quarto tag, and they're either from people that made quartal or those that are like us making slides of quartal and they do some neat little trick and I'm like I gotta bookmark that. There's a lot of these tips out there. A lot of them are quite scattered, but there's been one person at the forefront of collecting these in such a way that we could benefit as a community. And boy, he has taken that even up a 100 notches here. So we are speaking of the newly released online book called slide crafting, authored by who has been one of the ones at the forefront of this.
Emil Fithfeld, who is on the as a posit engineer at Posit does a lot of work with the the tiny models team, but he has also been making so many great resources about Quarto and in particular making slides of Quarto. His blog post that he called slide crafting had been featured as previous highlights before but now he has assembled them into one place called the slide crafting book and this is still early we always saw this come out I believe about a week ago or so it was after comp for sure and it's got a collection of all these tips that have been scattered or he has mentioned in his previous blog post but it's of it's not feature complete yet but what it has here is of definitely enough to get you started on the right foot.
All with kind of a crash course of creating your own custom theme for your coral slides in ten minutes or less. It literally has from start to finish what that process looks like, and then the rest of the book goes a deeper dive into each of these. What goes into the theme? Right? What colors do you select? What kind of fonts are you using? And how big or small are certain elements? Also, like I mentioned, what is the CSS styling associated with it? All of these are dedicated sections in this book, and I've already, just looking at this, have learned so much already that I'm gonna take into my next portal presentation.
And that's just the theming side of it. There's also the content block where we're gonna look at different ways that you can display your code, different ways you can highlight certain elements in your deck, changing layout. Where do things go? Left, right, moving it up and down? All these things that you can do with Quarto are all documented here. Like I said, it's not quite done yet. It's even got, other sections such as when you're showing a code demo using things like ASCII cast to help with that kind of recording like setup of a code chunk, ways you can bring interactivity via fragments and highlighting certain blocks of of of of elements, changing the styling on the fly, and even the the start of an extensions, section which is still not there yet, but I am looking at I'm gonna keep watching this quite closely.
There's a lot here. I've always wanted something like this because I want something where I can go to just in one bookmark, get all these tips. At some point, a lot of it will become muscle memory. It hasn't yet. So every presentation, I've learned something new. I do something different. In the past one at POSITCONF, I did the the highlighting of the code and narrated through each of those sections of, like, a shiny state block, but there's just so much. There's so much to learn. I think this is the start of something huge here. So, yeah, Is the slide crafting? Yes, it is, and I am here for it. What do you think, Mike?
[00:20:58] Mike Thomas:
Yeah. I I can't agree more. I am slightly upset that this came out how many weeks after Positconf?
[00:21:05] Eric Nantz:
I think maybe a week after.
[00:21:08] Mike Thomas:
Oh, man. I could have used, like, so much of this in my presentation. This is this is fantastic. And it's so many it answers so many of the little questions that I've had and just kind of not done the deep research into to figure it out, like image placement, layouts, things like that that are just, these great quality of life improvements for your presentation that I knew I needed to incorporate, but just didn't have the the bandwidth or didn't necessarily have a resource like this that I could go to to quickly get answers to my questions. And don't get me started on what the LLMs fail to know about, Quarto, reveal JS at this point because I I tried and and struggled quite a bit with some of the responses I was getting. But, anyways, this is it's just gonna be this is almost like a big book of type of a resource, and and I it's not super dense either. I mean, there's a ton of content here, but it's pretty easy and well organized to get to what you might be looking for. And even if you're not looking for something specific, you know, like I did, in preparation for the show today, I found myself, you know, scrolling through every section, you know, taking two minutes per section and and picking up something, constantly in terms of, theming and SCSS, which is, I promise, not as scary as it it may sound if you're not familiar with CSS or or SCSS.
It's maybe a little different to look at at first, but the amount of control that it gives you to incorporate your, you know, companies, or your organization's brand and logos and colors into your slide presentation that that that go a long way, believe it or not, to getting folks to pay attention to what you're presenting to them. That all everything is essentially right here. And some fantastic little things that I could have certainly used. I had, some slides in my presentation that I needed to kind of manually we had a clicker that we were holding, but I needed someone may out there may have the same type of situation. But some had elements in the slide that I needed to scroll down, and your clicker can't necessarily do that. You would have to go to your mouse or your touchpad on your laptop to be able to do that. It's not just advancing a slider, not advancing a slide. Well, of course, ML has a section in here, I believe, under, fragments potentially that shows me exactly how I could have done that.
You know, advance a tab set or change tab sets, I should say, or scroll, on some some content that's on your slide through, you know, the back and forward, buttons in the presentation. Oh, man. I could've used that so much. But, anyways, just a fantastic resource. I can't even pick sort of one area to highlight because I was just picking up stuff everywhere throughout this book. So I'm excited to see how it continues to expand, but a ton of content here, a great start. I think it's easily digestible. And if you go to this URL and don't find something, that, you know, you learned that you didn't know before, I'll give you a dollar. It's how confident I am. This is amazing here. And one thing I'm
[00:24:35] Eric Nantz:
as I'm assembling my notes through various presentations, even if I see a potential to hopefully add maybe to this, I'm hopeful that this will be a community driven effort as well. I may have some things to add around the video, playing when you get embed videos in coral. There's some neat tricks you can do with that as well, which were really handy for my talk with the the the Mega Man inspired app demos that went without a hitch. Thank goodness for that because I was a late night hack to get there, but that that that would be another time. But, yeah, even you, Mike, you utilize videos nicely in your part of the presentation and do a poll request and and issue notes. So, yeah, I could see. I yeah. This is definitely growing, but, boy, there is enough here now that for your next quartal presentation, whenever you're doing this at the day job or you have an external conference you're presenting at, I think you're gonna have enough here to do a a terrific job from both the UX side of it and, like I hope, have fun doing it because it is a lot of fun to make these.
Well, if I'm the curator for our weekly issue, you know I'm gonna have something with Shiny involved in it and, and, and, podcast. So, luckily, this was voted by others as well, not just me. But especially at Pazitconf and many avenues before this, we are seeing very much the intersection of large language models to enhance both development but also giving unique experiences to Shiny apps that just simply were not possible even just a couple years ago. There has been a terrific series that kinda takes you from literal start to finish, so to speak, on how you might approach building LLM AI model capabilities in your next Shiny app. And so we are talking about the third in this series called the shiny side of LLMs, and this has been authored by Vera Elfink van Leeenpoot who has been a prolific member in the shiny community since near the beginning.
She has done tremendous resources. She's also doing courses about Shiny as well. We have a platform called Athletics that I've heard great things about, but in any event she has been embarking on this journey to bring up to speed somebody who's new to the world of language models, what are the nuts and bolts behind how it works, but now we're at the third part where we actually plug this in to a shiny app. So the first parts are talking about what elements actually do and what they don't And then the next part talked about going from prompts that you put into the LMS interfaces and how the responses are constructed.
Like I said, now it's time to put this in action. And this comes full circle when we last talked about because the use case for this is going to build a Shiny app that will let you upload the contents of your quarto presentation and get feedback from the LOM on what's working well and what might be tweaked. So it is in essence a peer reviewer of your next portal slide presentation. Oh, boy. You told all things I wish I had had sooner. Maybe I could have used this before the conference talk, but nonetheless, better late than never.
So she's calling this app deck check, and it's gonna start with some narrative here and then the post about first why we choose Shiny. Well, I think the question is why not, but that's another point for another day. The point being is that within the ecosystem, now we have packages both on the our side and the Python side to integrate these chat like interfaces into your app this is called shiny chat on your side and chat last on the Python side so once you have that ready to go and and install either of those packages for your use case, you've now got to choose the LOM provider, of course.
I've been a big fan of Anthropics Cloud Code. Their models have done really well for me, but, nonetheless, whichever either hosted platform you choose, you're likely gonna have to get an API key because this is gonna go into your configuration based with the Elmer package to make all this work. But like I said there are code examples here to build this app literally from beginning to end and we start off with getting the chat interface going outside of shiny so I'm gonna focus on the r tabs here in this what looks like a cordial blog post with Elmer and configuring the chat tie into anthropic so there's a chat underscore anthropic object that you create where you just say the model the system prompt that you give it, again, plain text, and any additional parameters specific to the model itself.
And then this is something new. It's not just the chat object, but you want to plug this in the Shiny. Right? And for this, you want to be able to kind of have that nice streaming effect as the response comes in, and that's where Elmer comes with the stream method of this chat object. And then that's where you feed into it some language that you want displayed in the app itself, and then be able to tell the LOM what to do with that. I've never played with the stream object before, so I'm gonna or stream methods. I definitely want to play with this after after this, after this episode probably.
It does some unique generator based on the coral package. I believe that's a way to kind of do the looping correctly, but this is this is really neat. So it gives you that natural look and feel when the LOM is responding back after the user uploads their coral presentation. The other thing to keep in mind is that you don't want to have a bottleneck in your app as this response comes in. You want to take advantage of what's called asynchronous processing. This is great for the UX of all this so that your chat doesn't grind things to a halt with the rest of the app as the response comes in that's where it's a combination of again the Elmer package and in this case the promises package on the our side to do the special chat underscore async function.
So this is flagged to do asynchronous mode for this and it returns what's called a promise object afterwards that will be rendered or resolved when the app needs it, but it's not gonna stop everything else. So with that, you've got kind of the building blocks in place, and now it's time to build the UI. This is where, for preparing my talk at the conf, I did a, vibe coding session to prepare the UI in my demo and I got like 90% of the way there. But here she's done the hard work for you. A simple layout would b s lib, got a nice of sidebar or page fluid layout, I should say, where we've got a module for the chat component and a header at the top kinda at, like, a little placeholder for the user to ask anything about the presentation.
Very simple UI. Again, a module chat mod UI is gonna drive most of this from the Shiny chat package. On the server side, it's making that connection to the anthropic model and then rendering the server side of this chat interface, chat mod server. They've done the hard work for you. This is a module plug and play. You don't have to reinvent the wheel here. Really great to see the progress on this because I saw the early days of it. It was a lot of custom code. Now, right off the shelf with ShinyChat. So once you have that, then you've got the scaffolding of a mess of a new UI. It's very straightforward right now. We're just seeing, does this work?
And then you see in the screenshot here the prompt that was added as part of that metadata going into the chat object and then the response from the LOM based on the title that the user entered they call it the shiny side of LOM so what's your feedback and it's got the response there so we know the the plumbing works right now it's time to make the app look nice we can't just do that one column interface right so now she does a little whiteboard in here a sketching here of what the app will look like, a nice sidebar layout with cards that are added to the application. We got a main content and and then there's a nice animated screen of what they're working towards, the app in action.
The LOM prompt is on the left, the chat interface, there's a nice little CSS loader icon, and then once it's done it's got a whole bunch of like a series of value boxes at the top. Things like length of presentation and other content, and then the actual advice that it has for the the given presentation with various pieces of feedback. So all the code is here for both R and Python, and you get again this very nice UI and the app itself is doing a lot under the hood. Right? It's actually taking that uploaded quarto file and then it's running quarto under the hood to get the HTML version and then it's getting that that HTML file which is then fed to the l o m based on what they've added in in the back end of it. So this is a nice kind of diagram showing how this process works but in the end you can copy this app right from the blog post and try it out for yourself providing you've got that key to anthropic or whatever AI service you want to you want to do here. So there's other enhancements that she talks about here, such as a loading experience where you get that nice little loading icon when things are happening. So the user's not left with just a white screen doing nothing.
Those nice value boxes at the top, nice refinements. Again, just giving that extra nudge, that extra pizzazz to the user experience. But there's a lot more to this. There's styling. There's how you handle error checking. But this is as comprehensive of a post I've ever seen to take one of these apps from start to finish. So there's some nice nice examples, like I said, for error checking. And then when you're ready to go, there's a great section, what you would do to deploy this. Obviously, there are platforms like Pasa Connect that would be able to take this, Shiny apps IO, the internal I should say the internal Pasa Connect or the external Pasa Connect cloud.
But if I dare say so myself, another way you could host this would be containers as well and plug that into whatever app, you know, of service a, lets you host a container. Again, you build the app. You've got many choices where you host it. Either way, as always, if you're gonna put something like this for public facing, wherever internal your organization or in the public domain, be wary of the usage of that API key because you don't wanna break the bank too. But in the end, for a small app that doesn't have more than a handful of users, I doubt you'll you'll do very much.
Add Jason to this post. I did I had a colleague ask me, hey. You've been using Anthropic for a while. What's been your cost of this? Looked back at my cost. I kid you not, Mike. From January to now, my cost for anthropic has been a whopping $3.20. It's time you did some explaining. That's less than a lunch at the cafeteria. So I dare say I'm doing things right, I hope.
[00:37:00] Mike Thomas:
I don't believe I believe it.
[00:37:02] Eric Nantz:
Alright. But yeah. Yeah. That's so that's where just from little me developing and using it with Shiny or Positron assistant and in some very low level apps that that's pretty tangible. But, again, fantastic series of posts here. This is a great, great example of taking the concepts from the first parts. Now put it in action for a literal use case that I most definitely will be using before my next presentation. But, Mike, you've been we've been watching this space for a while. What did you think of Verway's exploration
[00:37:35] Mike Thomas:
here? It would be one thing for Vira to just, you know, use some sort of toy example and go through the entire process of creating both R and Python Shiny apps with the LLM capabilities in the back end, the ability to render the Quarto document. Doing that all in just one language would be incredible. She's done it in both languages and with a use case that is incredibly, I don't know, like, relevant or or tangible, practical, for anybody that's having to make a quarto presentation, which, you know, we just talked about in the last blog post and we've both done recently. And by the way, your presentation at Pazitconf was was fantastic. I think you gave me a little too many flowers there, compared to your presentation. But, this is, as you mentioned, probably one of the most comprehensive blog posts I've ever seen. I think it's well deserved that it's on, Pozits, series of shiny blog posts. And a I'll just pick out maybe a couple things that I I really like.
I think the error handling is something that we talk about a lot, Eric, and ensuring that your users aren't suddenly just seeing, like, red letters and error messages that are cryptic on their screen, and you do your best to try to this is an approach we always take is maybe pop up a modal when there is an error that tells them, hey. Something went wrong. Maybe you're able to to provide some pass or if statements that categorize what went wrong to give them a message, or maybe you even just voice the the message itself, you know, that you would see in the the terminal in that modal, but it's a a much better user experience than the whole app essentially just breaking. Right? We certainly don't want that to happen, if possible.
The the UI and the design here, I appreciated the time that she took to wireframe, what she wanted to put together, before that. And it's also you know, I don't think this is the purpose of the blog post, but even just some of the ways that she went about crafting her her prompt, yeah, a system prompt, if you will, to sort of instruct the LLM what it should do, you know, how it should respond and things like that were really interesting to me. She does have a system prompt in there, and then she has sort of a a more verbose regular prompt.
And the results look fantastic. I'm very excited. I have my cloud API key ready, and it seems like I can just sort of copy and paste this this code locally and try to run it myself. Myself. And I'm quite sure I'm gonna be blown away here, but a fantastic deep, deep dive into the art of the possible with Shiny, for R, Shiny for Python, and their associated LLM, counterparts these days.
[00:40:37] Eric Nantz:
Yeah. I am, literally, this afternoon, gonna build a custom next dev environment with Rex just for this because I am that eager to try it out. So, absolutely, that is on my on my list here. And, again, this was the third part, the culmination of a three part series. So definitely, like I said, if you're new to this, definitely check out her existing material. It does a great job of breaking down what can be very ambiguous terms if you're new to this. I mean, even me personally, I still get tripped up by some of the terminology thrown left to right on LinkedIn posts or whatever else in conversation. So I definitely like to have it explained to me practically, but not not in a condescending way. Verily does a terrific job with this. So, yeah, this is this is right in my wheelhouse, and, yeah, I'm doing still doing chordal slides. Might have to do one again before the end of the year, so this is this is gonna be utilized for for darn sure. But, we hope you also utilize the rest of the r weekly issue. We are running a bit strapped for time so no additional fines here. I do well, maybe selfishly speaking, I do have in this issue my adventures with switching the shiny dev series site to corridor so apropos of what we talked about today lots of interesting findings there and I'm back to blogging now. I got a blog component of shiny dev series so, who knows how long it'll last but I'm gonna try and be diligent with it, but it was it was it was good to get get the creative juices going again. So that's in there but also many many other things too. So with that we will, wrap up here but, of course, we love hearing from you in the community and we got many ways to get in touch with us. You can send a poll request.
That's time that's definitely appropriate after what I went through this morning, a poll request to the markdown files that comprise the our weekly project, especially the next issue. We have a draft markdown file. If you find a great resource, new package, great blog post, maybe you wrote a great blog post you want it featured, yeah, just send a poll request to that draft issue template and the next curator will be glad to merge it in. And, also, you can get in touch with us on the social medias out there. You can find me on Blue Sky these days. I am at r podcast dot b s k y dot social.
Also, I'm Mastodon at [email protected]. That's a mouthful sometimes. And I'm also on LinkedIn. You can search my name and you find me there. Mike, where can they get a hold of you? You can find me on
[00:43:10] Mike Thomas:
blue sky at Mike dash Thomas dot b s k y dot social, or you can find me on LinkedIn if you search catch Brook analytics, k e t c h b r o o k. You can see what I'm up to.
[00:43:23] Eric Nantz:
Awesome stuff. Then again, big shout out to all the listeners out there that said, hi, nose back at the conf. I will always appreciate all those kind words. We're trying to get back to the regular schedule here where we're doing our part, trying to anyway. But, nonetheless, we'll close-up shop for episode 100 212, I should say, of our weekly highlights, and we will be back with another episode of our weekly highlights next week. Hello, friends. We are back of episode 102 212.
[00:44:02] Mike Thomas:
Well, they so I had my speaker notes, you know, on one screen Right. And then my slides on the other screen. And I thought that if I
[00:44:10] Eric Nantz:
pressed play on the video on my speaker notes that it would translate into the video. They're separate. They're separate. I found out the hard way you know before. Yep. Oh, man. Okay. I didn't know he had videos in yours. I would have told you, but No, I know. I'll have to check yours out. You're good.
Episode Wrapup
Uh oh ... Outtakes!
