Over on the CosmoQuest Discord, I’ve been answering some questions about our software, and I wanted to explain what’s going on in more detail. This post is geared at the tech heads among you, and it’s a bit of a “history of the making of”, and a bit “how the sausage is made.”

Right now, CosmoQuest is running two radically different versions of our Citizen Science Builder (CSB) software. The version on this server, cosmoquest.org, is our new CSB v6, which is build in Python on a Django framework. We were about 70% done with this version when we lost our primary funding and with it, the bulk of our software team and all senior developers. We have finished it off with a great and recently graduated student – Herlayu / Tanner – who rose up through CosmoQuest, and a (paid!) intern – TimT. I’m proud of what of we’re creating, but it is still a work in process.

Our Bennu Mappers project, bennu.cosmoquest.org, is running a stand-alone version of CSB v5. This version (5.2) is based on PHP using a Laravel framework. This is the version we had been running in recent years on CosmoQuest. Some of its libraries are a bit old, and some of its styles are a bit crunchy, but it is (mostly) rock solid. (It likes to eat the occasional sacrificial image – we’re working on that.) We’ve turned off all the bells and whistles, and have asked it to do one job – let you map Bennu and store that data in a way that matches the mission’s data reduction pipeline.

The two systems have slightly different database structures, and this is why you have to setup a login just for Bennu. (We will be allowing folks to import their data to CosmoQuest’s main database after July 10.)

Some history…

When we first launched CosmoQuest back in January of 2012, our goal was to become the WordPress of Citizen Science. That is still a goal, but how we’re going to get there has been a bit of a rambling journey. We started out as a straight PHP project. PHP is a very flexible language that lets experienced PHP programers (hey, that’s me!) write zippy code for the web. Thus, we grew and evolved through versions 1-4 of our software.

PHP is not a language that is taught in university, however. This meant that when we hired students, they had the doubly steep learning curve of learning a new language and learning a massive code base. As long as we had three of us who were senior (Me, Cory, and the OG Joe), we were good. As money got cut in 2015, our staffing got cut to me and Cory, and we were ok, but I asked Cory to give the web landscape a good hard look and see if there was a better way forward. We had to acknowledge that the amount of time I was spending on administrative tasks was on the rise, and we had built ourselves a situation that wasn’t sustainable.

Iterative Design Interrupted

In 2016, we had a massive new grant, a tough timeline that we could match if we could hire the people in our budget, but… the Illinois institution we were at hit us with a bureaucratic wall (this was year N of Illinois having no state budget – everything was broken), and the senior position I planned to hire I couldn’t hire. With nothing but students and Cory … <insert primal scream> … and with 100% of my effort going to administrivia … <insert hair pulling> … we adopted Laravel. Our thinking was that a framework would make it easier for students to learn, increase the library support we could lean on, and it looked like Laravel was a fast growing community.

Nope. We were so very wrong.

(Added bonus feature: Our new layout was based on design rules from a third party consultant, and we weren’t allowed per terms of our funding to design based on what we thought would work best.)

So, in November 2016, we launched code we were okay with, and planned on iterative design to perfect it. This is web-design at its best. It’s impossible to know exactly how the public are going to react to a site until you put it out there, and most major sites do regular tiny design tests to see how things can be improved one repo push and pull at a time.

But in January 2018, we got word we weren’t even allowed to update our fonts without approval from NASA HQ and demonstrating that the changes we wanted to make were based on research-based best practices. This kind of restriction makes sense if you are developing expensive instruments or printed materials or things that require expensive teacher training every time a change is made. BUT, for us changes had no cost and were the norm of the discipline. This restriction effectively locked us into a design it turned out no one liked and we couldn’t change. The issue is simple: best practices in web design change faster than academic papers can be published, and I couldn’t point to research based results on why I should try A over B without doing my own A / B testing, but … we had no money to do A / B testing.

Oh – and our funding got massively cut in January 2017 when I switched institutions to escape Illinois’ crippling bureaucracy (the thing preventing me from hiring the senior person I needed), and because of the funding cut I couldn’t hire the senior person I still needed. Irony.

Sometimes you need Python

Cory, my hero, my right hand human for more than a decade, looked at me in winter 2017/2018 and said we needed to switch to a language the students would know. We gave up on finding a new senior person. He wanted his last act for CosmoQuest to be putting together a python version of our code. Working on a project where upgrades aren’t allowed is beyond frustrating, and with my blessing, support, and encouragement, he left us in July 2018 to work on creative things on his own. He had a complete draft of the Python code done when he left in the summer. We’ve been working, with only students, to finish things ever since.

Freedom with less Funding

Loosing our primary grant January 1, 2019 was actually a grace. I have updated our Bennu Mappers code 4 times today to make wording changes that reflected the wants and needs of our audience. Tanner, our awesome lead developer, has fixed a refresh bug and added some cool features. This is how it should work: Iterate, test, push, repeat. Working with the Bennu Mission on a contract has been an amazing collaborative experience.

We are just a tiny team so the python version of our code wasn’t quite ready for use with Bennu.

CSB v6, python edition, is a repo full of ghosts. The assets include echos of verions 1-5 of our software, and there are views for things no longer needed, and empty directories with names of ideas we realised were bad ideas and never built. These ghosts also include random developer keys, and other things that are safe with their file permissions, but really don’t belong in a repo and can’t be in a public repo.

But it’s our repo, with our ghosts, and we will love it and use it and iterate on it until we have something better.

Let’s code together

After I’m done with this blog post, I’m going to open a clean repo, CSB v7. I’m going to give it the ability to login, and maybe one or two other things, and then make it a public repo sometime next week. It’s time we work on creating an open source version of what we do, one feature at a time. We’re bringing in our own BinaryAblaze / Annie, and still working with Tanner and Tim, and a few new people here on our Build Team.

And you. I invite all of you to join us (sometime end of next week).

Together, we can do more than just map the solar system; we can create a better way to map our solar system, one line of code at a time.