MIDI keyboard animation demo

I've been playing around with UE4's possibilities for triggering events using MIDI (a language for communication between electronic musical instruments). At first, I thought there might be a Blender plugin for this, but the Python script I found online doesn't seem to work correctly in the latest version of Blender. 
Instead, I'm using the "Procedural MIDI" plugin by Scott Bishel available on the Unreal Marketplace, and a piano model I purchased several months ago. The keyboard that came with that piano had the keys fused together, so I had to separate them first in Blender, then import them into UE4, each with unique names for each key. The Procedural MIDI plugin allows the data from the MIDI file to be accessed by Blueprints, which in this case changes the position of one of the 88 keys. Several hours later, and after coding an extremely laborious Blueprint, I have this video:

Here is the basic Blueprint to run the key movements:

The first half essentially opens the MIDI file, and plays it. The second half siphons off the MIDI data as it is played, and assigns the note data to a single variable. A function in the middle (see below) translates that number to a specific key on the keyboard, then whenever the MIDI file triggers a "note on" event, the key referenced by the Receive Note Name function is rotated by -3°; a subsequent "note off" event returns it to 0°. Here is that function (the "Are we there yet?" function), which goes through every note on the keyboard, and confirms if it is the one that should be moved or not:

And this is as much as could fit on the screen at once:

Believe it or not, this works because MIDI data is sequential, meaning no two notes actually play at the same time. The human ear perceives separate MIDI events as chords because it happens very fast!

Classroom props and game coding

Working on the to-do list this past week meant creating some props in classrooms and re-doing some earlier models. Also, I tried to fix some bugs in my coding, which was only partially successful. 

Classroom desks, modeled after photos from the Rosemary and a vintage desk online:

A classroom full of desks, modeled by me in Blender

 

Classroom desks as seen in the Rosemary

Similar model of a vintage desk from an online auction site

The vents on the face of the stage apron: 

This photo from the 1923 Rosemary is the one I used to reference the wood wainscot, and more recently the vents

Finally, I spent many hours pouring over my code for two items: Gamepad functionality and the guided tour that unlocks when the player puts out the fire in the coal bunkers. While the guided tour will need additional work, I finally got the instructions for a Gamepad to appear on screen appropriately: 

I noticed in numerous games that if the mouse is used, on-screen instructions and icons show keyboard/mouse icons, but if the Gamepad is used, the same instructions switch to Xbox 360 icons. Unreal has limited Gamepad support, so all the coding has to be done manually to support a Gamepad throughout the engine, especially on menus. I finally made it so that the correct icons display at the beginning of the game, and in several of the menus. This may seem like a small victory, but it was very exciting when it actually worked! 

Additionally, I allowed both the left Gamepad trigger or the mouse wheel to zoom the field of view from 90 degrees to 30 degrees. This, of course, resulted in lots of fun zooming in on things!

Courier newspapers and Tower Ghost

First, Mayor Diane Marlin of Urbana was gracious enough to write a quick Facebook post earlier this month about my blog. 

Within the first 24 hours, the latest fly-through had over 1000 views. As of today, it has over 2400 plays. Through all the shares, comments, and re-shares, many people have expressed their joy at seeing the Project. I am grateful for everyone's enthusiasm, and I now feel especially encouraged to see UHS: 1914 to completion. Thank you to Mayor Marlin, and to everyone who watched the video, and chose to let me know how much they liked it. 

In an effort the add an educational gameplay element to the interactive version of the project, I added newspapers strewn throughout the building. Newspapers from the Urbana Courier-Herald are available online at the Illinois Digital Newspaper Collection, and are fully searchable through the magic of OCR. I found several issues that deal with the opening of the school -- particularly the delay because of the fire. Even though the papers are searchable, I found myself reading entire issues because the history contained is fascinating. 

Within the game, I set the newspapers to change once they are read, so the player always reads the issues in sequence. Approaching any newspaper in the building will display an option to read the paper, upon which the full-resolution page appears, and the applicable article zooms into focus. The player must extinguish the fire in the coal bunker in order to read the article about the school actually opening. 

Going back to read these again, I saw a detail about how smoke was pouring out of the manhole covers over the coal bunker, so that necessitated adding smoke. 

Getting the newspapers to change correctly took a lot of work, so I followed this up with something "fun:" the tower ghost. The ghost was not something originally on my list of tasks, but two developments encouraged me to add it. 

First, a piece of free software exists called MakeHuman. This originally was an extension for Blender, but now is a stand-alone application. The program allows easy creation of fully modeled and rigged digital people. "Rigged" means that the model can immediately be animated. Body types, faces, clothes -- all are customization. I used the software to generate an appropriately spooky-looking lady with a long dress. 

Second, the Carnegie Mellon University Motion Capture Database is available online, and the data has been converted to be compatible with Blender. Numerous videos on YouTube describe how to combine these animations in Blender with models created with MakeHuman to make game-ready assets. I found an animation of someone walking up stairs, and combined that with the woman I generated from MakeHuman. 

Imported into UE4, the addition of some lighting, sound, fog, and a touch of post-processing makes the effect quite convincing (and legitimately frightening!). She appears in a special, random sequence. Rather than spoil the surprise, I will only post a still screenshot here. 

Spooky!

New flythrough with Gym addition

I posted this to YouTube on Christmas, but only now got around to writing about it. The November video (v7) was slightly retooled, and a new video of my current progress on the 1918 gym addition was spliced in the middle. The transition is not perfect (it reminds me of the B&W-to-color transition in Wizard of Oz), but it still gives a good idea of the current state of the project. 

Gym details - December '18

I have a list of tasks now, and the longest portion is for the gym. I tackled a few of these this month. 

UV mapping for gym roof (and improved ship-lap texture)

Old-style basketball court markings

Updated bead board texture for balcony

Rear pool windows

Completed pool model, including portion under water

Detail of the deep end of the pool

Photo reference for the pool from the 1923 Rosemary

Pool room with brick wainscot

Transparent motion water texture

908 South Race

In an effort to acquire more green space surrounding the school, District 116 started purchasing houses neighboring the school, and demolishing them. The house on the corner of Iowa and Race streets, 908 South Race, was one of these. Working only from a pair of reference photos, I modeled a reproduction of this house. Comparing it to the reference photo, it looks like it is not tall enough, so I may have to go back and correct that. 

Reference photo from the Champaign County Archives

908 S Race, recreated in Blender

Imported into Unreal 4.21

Fly-through version 7

Here is the new fly-through video! Although it's obviously not complete, the video shows many of the things added since February.