The KITS Project
I was told once by a friend of mine that Dr. Jennifer Kreie’s classes were a lot of work but that the rewards and benefits reaped by the experiences of her class projects were priceless. Dr. Kreie’s methods are thorough, and I think it is fundamentally impossible to take a class of hers without learning a lot of material. Our BCIS 450 class was certainly no exception, and the experience I took away from that course was both the most memorable and the most applicable to the real world. Since the project Dr. Kreie assigned was so vast, I have elected to provide significant details along with its very own page.
A Brief Overview
Dr. Kreie’s BCIS 450 class was structured such that half of the class time was spent discussing a variety of topics related to databases and systems analysis while the other half was devoted to teamwork with the goal of applying our knowledge to a real world problem. Each year, Dr. Kreie searches around for data-driven projects from the local community that are applicable to the coursework, gathers details on the requirements of the project, assigns teams, and provides the students with an environment to compete with each other for the best implementation. Since the project definition is often quite broad and requires significant data manipulation, it isn’t much of a stretch to imagine such a design applied to a much grander scale.
During the semester I took BCIS 450 (spring 2008), Dr. Kreie had gotten into contact with a local government organization responsible for cataloging threatened and endangered species. This agency was already using Microsoft Access to store some data they had collected over the years, but the ad hoc design was clumsy and much of it was imported with few changes from Excel spreadsheets. The database we received violated nearly every principle in database design and was denormalized to such an extent that it appeared no different from the collection of spreadsheets it was likely intended to replace.
The objectives for the class as a whole were relatively straightforward, but we had a great deal of work ahead of us to produce a worthwhile finished product. The first requirement of the project was to expose us to project management and planning; we had to create a schedule, statement of work, and apply the principles of rapid application development (RAD) to our project by introducing weekly sprints. These sprints were designed to tear out bite-sized chunks of the final project requirements and make it much more manageable to perform useful work, and at the end of the week we’d have something to show for our efforts. On the technical side of the project, we were also required to normalize the database such that it was no longer in violation of the first, second, and third normal forms. We were also required to document the features of our database and create a usable UI (also via MS Access) that was relatively accommodating toward individuals who likely had little knowledge or experience with databases. Once our design goals were met, the class was to critique each others’ group projects, and Dr. Kreie would select a “winning” project to present to the client (the government agency in this case). I believe the class was composed of about five individual groups ranging from four to five individuals, although I think our group was one of only two total that was comprised of four people.
The Project Structure
Our group was blessed from the start of the project, I admit; we were motivated and willing to succeed. I think those two issues are the greatest hurdles any project can face, but once a group is both motivated and willing to succeed, anything can be accomplished. Humans are a social species after all, and what one individual cannot do alone, many can.
Our group composition was among the smallest in the class. Most of the other groups were comprised of five members–ours had four. In a somewhat ironic twist, I felt that our performance was approximately three to four times higher than that of the larger groups. Again, I appeal to the previous paragraph to explain why I feel this was the case. Our members were:
- Kohl Kemp as Project Lead
- Anthony Matta as UI and QA Lead
- Hope Rodriguez as Database Lead
- Myself (Benjamin Shelton) as Documentation and Programming Lead
In all actuality, a huge chunk of the project was the responsibility of Anthony and myself. I remember spending countless evenings that semester talking with Anthony via AIM until the wee hours of the morning trying new things, tweaking, adding, changing, implementing, and working. I don’t mean to disparage the inputs of our other members (they did handle much of the administrative overhead and paperwork Dr. Kreie required, after all), but the most memorable moments to me were those spent working with Anthony–Hope started joining in on our chats, also via AIM, but that was a little later on in the project. Ultimately though, working with Anthony was a tremendous pleasure, and if I were to ever do it again, Anthony would be my first pick. He and I alone meshed very well together, and I honestly think that the two of us could outperform the work of ten or fifteen people. In fact, we did (though it was probably closer to twenty!).
It Takes Synergy
Not that I mean to brag; quite the contrary–were it not for Anthony, there wouldn’t have existed the strong positive-feedback cycle (or “synergy” as Anthony liked to call it) where we bounced ideas off each other and worked our butts off for weeks. We were proud of the final product and our accomplishments. The efforts were certainly well worth it!
Incidentally, Dr. Kreie informed me during the following fall semester that she had selected our project to present to the client. While it wasn’t without its flaws–we only had a limited amount of time to work with, after all!–it was of such quality that she felt it to be the best in the class. I feel much of that reward is Anthony’s; without his work on the user interface and the reports (oh good Lord the reports–I have no idea how he made them look so damned awesome!), the project would certainly not have appeared as attractive as it was. Looks matter–data and backend be damned!
(Yes, the back end is important; it’s just not the first thing people see or even notice. Ideally, you don’t want people to notice the back end!)
The Early Days
During the earliest moments of the project, I recall each of us scrambling to get our ducks in a row. There was a fair amount of paperwork required by Dr. Kreie before we could get started. Not that it would have mattered much if the paperwork were late; unfortunately, it wasn’t until March (or maybe late February) when the individual who was supposed to provide Dr. Kreie with the project requirements and sample data finally got in touch with her. The government agency was understaffed during that part of the year, and it was obviously very difficult to communicate with the “client” on short notice. This later turned out to our advantage as it gave us time to carefully consider and plan our questions to make the most of what precious little time we did have.
It was my understanding that during previous semesters, Dr. Kreie required the students to travel around Las Cruces meeting with their “clients” and spending as much time as necessary to understand their requirements. We didn’t have that luxury, of course, and when we did manage to interview the agency’s contact, we were required to use standard classroom time. I think we did a respectable job given the resource limitations, and we were certain to ask as many questions as possible that would impact the final design of the project. It was something of a one-shot, one-kill sort of project; we didn’t have a second chance to get things right. Everything had to be done correctly the first time. We simply didn’t have time enough for second chances.
That’s when the stress started to mount, not that we were particularly stressed out. We were hard-pressed for time but never stressed out. In retrospect, I realized that this particular project smacked more of the “real world” than anything else I ever did before or since in college–it was that good.
Once we had an opportunity to talk with the client, we presented our ideas and design. The design, of course, was Anthony’s realm, and I recall that he had worked on it over the course of several days to perfect it. The interface was a prototype and hardly representative of the finished product, but the important part was that it looked good. I was certain to ask our contact what the scope of their data requirements were, and how they intended to have the data imported. I believe we were fortunate in this regard, as much of the fundamental data was already available in the samples we were given–they (the agency) would fill out the rest. Naturally, much of the data was proprietary information and we weren’t allowed to see it, but the scrubbed data we were granted access to was sufficiently detailed that we could base our project off of it with little concern about any underlying changes. Plus, the data we did have access to was of historic importance, and they wanted to maintain it as best as possible.
In the end, I think we had access to something on the order of 90,000 entries in one table, and a good 100-200 thousand in another. There were a bunch of other interspersed entities here and there of moderate size, but it is noteworthy to mention that the database was quite large. In fact, the copy we were given was nearly 100 megabytes in size decompressed. It turned out that the agency’s technical types must have neglected to compact the database to reduce its foot print on disk. Interestingly, MS Access is similar to SQLite in this regard: If large swaths of data are moved around or deleted, the free space isn’t actually “freed” until the database is cleaned up a bit. (I believe that’s how it works, anyway–SQLite has a similar issue when large tables are DROP’d.)
For what it’s worth, the compressed database was about 5MiB in size (in a .zip), and the compacted version was around 33-35MiB.
I remember a rather comical picture of the day we finally received the database from Dr. Kreie. Each group was huddled around a single workstation as if we were nursing a poor quality Youtube video. I’m sure that outward display of anxiety was the result of WebCT’s notorious instabilities or perhaps it was due to our expectations. I know we weren’t certain what to expect of the database (we were told it had a lot of data), but I don’t think we anticipated encountering a database of the size we were given. “How many species can there possibly be,” one of us quipped, half-joking as the 30-something megabyte file was slowly pulled from the campus WebCT host uncompressed.
Kohl had a few choice words for the process. I think he was hungry.
I’m sure we all expected WebCT to crash at some point during the transfer, too. It had been acting up that week since the first or second round of exams were being administered in full force around campus, and the WebCT service was notorious for its tantrums under load. Plus, we were eager and a little anxious–without the database, we wouldn’t be able to make our plans for the weekly sprint. Worse, we were already behind schedule precisely due to delays in acquiring the sample data.
I don’t think we were prepared for what was to come. You can imagine this in the voice of Illidan Stormrage, if you like.