Tuesday, August 30, 2022

Export Meetup.com Events to a Github Pages Jekyll site

For a long time, our main Omaha Java Users Group communication has been through meetup.com/omahajava/, however we do still have an ojug.org website too. At first, we would nicely copy the event to both Meetup and OJUG.org, however within a few months you can see that we simply stopped duplicating effort and the events on ojug.org quickly got out of date. Not a big deal at the time. But it would be nice to archive our past events to the ojug.org site (especially as we explore Meetup.com alternatives in the near future [1]).

(if you want the TL&DR: see all the details at https://github.com/jeffsheets/ojug-meetup-export.)

Exporting Events from Meetup.com

The first step was to find a way to export our past events. I had hoped for a CSV export through the admin UI, but didn't see anything. I then thought it might take some web scraping, or maybe some network dev tools api call watching. But noticed that there was a grayed out setting in the admin UI for the Meetup API 🤔. A quick google found the Meetup GraphQL API Playground page. And to my surprise, sending a test query just worked! 🎉 After a little trial and error from their API docs, and increasing the result count to 100 to get all of our events without paging, I was able to export all of the past events to JSON with:
  query($meetupId: String!) {
    groupByUrlname(urlname: $meetupId) {
      description
      pastEvents(input: {first: 100}) {
        count
        edges {
          node {
            title
            description
            dateTime
            going
          }
        }
      }
    }
  }
And some inputs of:
  {"meetupId":"omahajava"}
Which gave a nice JSON result, like this:
{
    "data": {
      "groupByUrlname": {
        "description": "Omaha's Java User Group [@omahajug](https://twitter.com/omahajug/). yadda yadda yadda",
        "pastEvents": {
          "count": 65,
          "edges": [
            {
              "node": {
                "title": "Angular JS for Java Developers",
                "description": "This month //etc etc etc",
                "dateTime": "2014-05-20T17:30-05:00",
                "going": 27
              }
            },
.....

ojug.org Tech

Before showing you how the event blog post pages were generated, a quick note on the tech for ojug.org (src at https://github.com/OJUG/ojug.github.io). It is running on a standard Github Pages Jekyll workflow stack. While we have thoughts to move that over to 11ty, for now Jekyll is working fine. Create a new markdown file in the _posts folder, merge to the main branch, and the workflow auto-kicks off and redeploys our ojug.org site like magic.

Generating Meetup event Jekyll posts

With this in mind, the blog needs .md files generated for each event in the events JSON. Using a little Groovy, this was done pretty quickly with a GSP template, some functions to prettify date formats, and a filename creation function. The groovy post.gsp template looks like:
---
layout: post
title:  "<%= longDate %> <%= title %>"
---

<%= description %>

(This past event was exported from Meetup.com)
(<%= attended %> people had RSVP'd to this event in Meetup)
Then the code that generates the template for each JSON event is in PostGenerator.groovy:
    void generatePosts() {
        def events = new JsonSlurper().parse(getClass().getResource(SRC_JSON)).data.groupByUrlname.pastEvents.edges
        events.each {
            def event = it.node
            def filename = makeFilename(event.title, event.dateTime)
            def outfile = new File("$DEST_FOLDER/$filename")
            def filecontents = new SimpleTemplateEngine()
                    .createTemplate(getClass().getResource('/post.gsp'))
                    .make([
                            title      : event.title.replaceAll('"', '\"'),
                            description: event.description,
                            longDate   : convertToLongDate(event.dateTime),
                            attended   : event.going
                    ])
                    .toString()
            outfile.write filecontents
        }
    }
When executed, nice markdown files are generated in the output directory, similar to 2014-05-20-angular-js-for-java-developers.md:
---
layout: post
title:  "May 20, 2014 Angular JS for Java Developers"
---

This month //etc etc etc

(This past event was exported from Meetup.com)
(27 people had RSVP'd to this event in Meetup)
The last step was to copy all of the new markdown files into the _posts directory, create a PR, merge it, and see the final results up at ojug.org!

Wrap up

And that's it! Writing this blogpost probably took longer than the process to export Meetup to JSON, and generate new markdown files for the Jekyll blog! You can view all of the full source on my github at https://github.com/jeffsheets/ojug-meetup-export

[1] - A quick footnote, about Meetup.com... Over the years Meetup has been great for advertising our group, attracting new members, having a great user interface, and easily collecting RSVP's for events. We've always been lucky enough to have great sponsors to pay the ever increasing fee, which is now up to $197.98/year. However Meetup's decision to not allow us (or any group) to "freeze" the account, means that our sponsor has been paying that fee for 2.5 years with little benefit. Talking to other local tech meetup organizers, it became apparent that many of us are pondering ways to free ourselves from these fees. Our sponsors could throw some pretty great user group parties with the savings! There's a lot of functionality we'd have to replicate though, so I'll leave that full discussion for another time...

Tuesday, May 11, 2021

Mock Intl and Date globals in Jest

In Javascript land, mocking the browser global objects can be a bit of a pain for tests. Searching StackOverflow gives plenty of complicated answers. Some suggesting using 3rd party mock libraries. Some that overwrite the global object itself.... But Jest already has this capability built-in and it isn't so bad:

So let's say you have a method that gets the user's timezone or the timezone offset. (the timezone offset is used sometimes since IE11 doesn't support easily reading the timezone, but I digress)

Now to test this, we'll need to mock out both the Intl and Date Javascript globals. We can do this using Jest's spyOn method to temporarily replace the global method with our own implementation. Notice that we setup the spy in the beforeEach and reset everything in the afterEach. The setup works something like this:

But that's it! No need to import an extra library. This is all supplied directly in Jest itself!

Tuesday, February 09, 2021

Blogger CSS Grid Responsive Layout

Yeah, a post about how to build a blog. Because owning a blog means that at least 20% of your posts need to be about the tech used in it's own creation. But this one is different. It's the tech used to upgrade a legacy Blogger site to a more modern responsive layout 😃

Why?

Well, while simultaneously an item of pride and embarrassment, I've had this Uncommented Bytes blog running for 17+ years somehow. (And more like 18-19 years but I lost the first bit of it when jRoller, the best hosted blog engine, went under and required the original migration to Blogger.) 

I wrote a post for the first time in a while, and was pretty annoyed with how it looked on mobile

https://twitter.com/sheetsj/status/1352434228483514371

I originally had a green-ish theme, and thought I'd tweak it to be responsive. But even though it was simple-ish, it was using some images for backgrounds and shading that were just overwhelming to try to update in the super-wordy blogger html template. (look at that beautifully retro Google+ link)

https://web.archive.org/web/20181227163052/http://uncommentedbytes.blogspot.com/

So I decided to "upgrade" to a cleaner look theme in Awesome, Inc --- which despite the genius name, was lacking in a couple areas:

  1. Fixed margins -- meaning not responsive to browser width
  2. No mobile friendly view -- mostly because it wasn't responsive, but also because I had used a Blogger switch to render a super generic mobile view too (which I think was triggered by user agent headers, yikes)

How?

Trial-and-error isn't fun to describe, especially when it mostly involves browser devtools CSS hacking. But basically I took the super-wordy HTML template and hacked in some CSS Grid layout. 90% of the hack-magic was accomplished with this bit:

        div.content {
            display: grid;
            gap: 20px;
            grid-template-areas:
                "header"
                "content"
                "sidebar"
                "footer";
            max-width: 100vw;
            margin: 0 auto;
}

Plus a tiny bit of responsive handling:

        @media (min-width: 800px) {
            div.content {
                grid-template-columns: 2fr 1fr;
                grid-template-areas:
                    "header header"
                    "content sidebar"
                    "footer footer";
            	max-width: 1280px;
            }
        }
        @media (min-width: 1024px) {
            div.content {
                grid-template-columns: 4fr 1fr;
            }
        }
        .gist .blob-code-inner {
            white-space: pre-wrap !important;
        }

And the last 10% was a few margin and padding tweaks sprinkled in other places. You can see the full code in my Github blogger-css-grid-layout project. And just in case I redesign again in the future, the full inception final product looks like this today: