Mark Johnson

    Live charting - A Clone Hero side-project

    Thursday 23rd April, 2020 - 12 min read

    A few weeks ago I decided to chart a few songs for Clone Hero. After a quick search I downloaded Moonscraper to create the charts and I was pleasantly surprised with how easy it was to use. I remember using Editor on Fire to make GH2 customs back in 2007 which was a fairly painful process, but using Moonscraper was a pleasure. What makes it even more impressive is it’s built and maintained by a single dev, very cool.

    However, when I started charting songs, one thing bothered me. I kept scrubbing backwards and forwards, playing and pausing to place a note before rewinding to play back the section and check the timing. I had to repeat this process for every note. While standard verses and chorus with consistent note timing were pretty quick to chart, slow sections with inconsistent note spacing required a lot of manual back and forth. I wanted to simply play along on the guitar and have the notes recorded into the chart as I played.

    With this in mind, I decided to try and make it a reality and got to work on my weekend side project.

    TL;DR: Not interested in the process? Skip here to see the end result.

    Step 1 - Building the project

    The first step was building Moonscraper. Finding the project on GitHub was easy enough and I proceeded to download and install Unity 2018.3.0f2. I opened up Unity, loaded the project and was greeted with a familiar looking highway object - off to a good start.

    Unity with Moonscraper project loaded

    I hit “Build and Run”, bracing myself for errors, but soon a folder popped open with the Moonscraper build files.

    Moonscraper build folder

    I double clicked the .exe and bam, Moonscraper was running.

    Moonscraper build folder

    20 minutes in and the project was already building, not bad. Time to start investigating the code.

    Step 2 - Understanding the entry point

    The next step was to find the entry point for where we would add our functionality.

    I clicked around the project files in Unity and found the code files lived in the Scripts folder. I could view the C# files in the Unity Inspector view but it became clear I needed a proper IDE so I installed Visual Studio 2017 and was ready to start exploring.

    Going back to the Moonscraper application, I wanted to behaviour of the Start Gameplay mode.

    Start Gameplay

    Start Gameplay allows users to playtest their charts. Moonscraper essentially operates like Guitar Hero, letting users play along with their chart and tracking notes they hit and miss. Within this gamemode I should be able to add functionality to read the input and write this out to the chart to allow the ‘real-time charting’ process to happen.

    I searched for StartGameplay across the Visual Studio solution.

    Find results

    This returned five references across four files:

    • ChartEditor.cs appeared to be the Main Application Window. It handled Update() and OnApplicationFocus(), global commands like Save() and Load(), and most importantly the StartGameplay() method we were interested in. But this method appeared too high level, and was responsible for managing the system state with ChangeState(State.Playing, ...). Our changes would go elsewhere.
    • MSChartEditorInput.cs contained a list of enums including StartGameplay. Not relevant.
    • PlayingState.cs managed State based on the action enums. Not where our changes would go.
    • This left us with EditorState.cs. EditorState.cs contained:
    if (services.CanPlay())
    {
        if (MSChartEditorInput.GetInputDown(MSChartEditorInputActions.PlayPause))
        {
            editor.Play();
            return;
        }
        else if (MSChartEditorInput.GetInputDown(MSChartEditorInputActions.StartGameplay))
        {
            editor.StartGameplay();
            return;
        }
    }

    This handles the “StartGameplay” entry point from the Moonscraper UI.

    To check my understanding I added a breakpoint, hit Attach to Unity, and waited for Moonscraper to boot up.

    Attach to Unity

    Unfortunately this didn’t go to plan.

    Unable to debug through Visual Studio

    Visual Studio wasn’t able to link the breakpoints to the running Moonscraper application.

    I didn’t want to spend more time on build setup. Not wanting to be put off, I decided I could make do with a basic log file. It would make debugging harder but it would do.

    private void WriteToLog(string stringToLog)
        {
            var logging = true;
            if (logging)
            {
                System.IO.File.AppendAllText(@"C:\Users\Mark\Documents\build\Log.txt", 
                                            stringToLog + "\n");
            }
        }

    With the above lines of code written, we had basic logging in place. I added logging in the if (MSChartEditorInput.GetInputDown(MSChartEditorInputActions.StartGameplay)) statement, built Moonscraper, hit Start Gameplay, and saw my print statement written to the file. Code entry point confirmed, good news. Time to dive deep to see where to add code.

    Step 3 - Finding where to add the code

    I read through the code. I searched the code for State.Playing to find which functions listened to Gameplay mode and I came across the Scripts\Game\Gameplay folder. The GuitarNoteHitAndMissDetect.cs file in Rules sounded particularly promising for reading user input.

    I found the Update method which captured user input.

    public void Update (float time, HitWindow<GuitarNoteHitKnowledge> hitWindow, 
                        uint noteStreak, GuitarSustainHitKnowledge sustainKnowledge)
    {
        // Capture input
        bool strum = GuitarInput.GetStrumInput();
        int inputMask = GuitarInput.GetFretInputMask();
        if (inputMask != previousInputMask)
            canTap = true;

    I wanted to see if I could write to the log file when the strum bar was hit:

    if (strum)
    {
        var log = time + " Started\n";
        WriteToLog(log);
    }

    With the following code I built Moonscraper, entered Gameplay mode and hit the strumbar. I checked the log file and I had an output:

    Writing to log

    Success. Who knew viewing a log file output could feel like an accomplishment. An hour in and we knew where to add the code.

    Step 4 - Adding the code

    Now we needed to read the input and output this to the .chart file.

    The input mask, int inputMask = GuitarInput.GetFretInputMask();, held the user input.

    As input was well defined, I started investigating the output.

    In Moonscraper, Notes are written to the chart when the user has the Note tool selected and clicks on the highway or presses number keys with Keys Mode enabled.

    I followed the usual Ctrl+F process, searching for Keys Mode and Note to understand where they were called, and found this:

    public class PlaceNote : PlaceSongObject {
        [...]
        protected override void AddObject()
        {
            editor.commandStack.Push(new SongEditAdd(this.note));
        }

    This looked useful but I was concerned it couldn’t be called from GuitarNoteHitAndMissDetect Update. Fortunately I found a ChartEditor Instance could be created with ChartEditor editor = ChartEditor.Instance which provided the editor.Push... capability.

    Now it was important to understand the editor.Push command to replicate this logic here. Note is a sub class of many objects that can be placed on a chart, such as star power, sections, BPM, and more. A Note object can be instantiated in several ways but for this scenario a GuitarFret object was needed which required a timestamp and a note.

    Now that writing the note was well defined, I went back to understanding how to transform the inputMask into a valid Note.

    Following where inputMask was passed to, I found:

    foreach (Note.GuitarFret fret in EnumX<Note.GuitarFret>.Values)
    {
        if (GetFretInput(fret))
            inputMask |= 1 << (int)fret;
    }

    inputMask was using bitwise logic.

    inputMask needed to be mapped to a Note.GuitarFret object to create a Note object to write to the chart.

    Following the inputMask variable from Update to UpdateNoteKnowledge and ValidateFrets, I found the logic below.

    public static bool ValidateFrets(Note note, int inputMask, 
                                     uint noteStreak, int extendedSustainsMask = 0)
    {
        inputMask &= ~(extendedSustainsMask & ~note.mask);
    
        if (inputMask == 0) {
            return note.guitarFret == Note.GuitarFret.Open;
        } else {
            // Chords
            if (note.isChord) {
                // Regular chords
                if (noteStreak == 0 || note.type == Note.NoteType.Strum) {
                    return inputMask == note.mask;
                }
                // HOPO or tap chords. Insert Exile chord anchor logic.
                else {
                    // Bit-shift to the right to compensate for anchor logic
                    int shiftCount = 0;
                    int shiftedNoteMask = 
                        BitshiftToIgnoreLowerUnusedFrets(note.mask, out shiftCount);
                    int shiftedInputMask = inputMask >> shiftCount;
    
                    return shiftedInputMask == shiftedNoteMask;
                }
            }
            // Single notes
            else {
                int singleNoteInput = inputMask >> (int)note.guitarFret; // Anchor logic
                return singleNoteInput == 1;
            }
        }
    }

    Pretty complex, particularly BitshiftToIgnoreLowerUnusedFrets. I didn’t want to learn the intricacies of dereferencing the inputMask, so I decided to simply print the value of inputMask to a file while I strummed out several fret patterns in Moonscraper. Looking at the log file I saw values of 1, 2, 4, 8 and 16 that corresponded to the five frets, with chords representing various other numbers.

    Happy with this I wrote a simple mapping function to capture these single fret inputs.

    // map input to frets
    var fret = Note.GuitarFret.Green;
    if (inputMask == 1) {
        fret = Note.GuitarFret.Green;
    } else if (inputMask == 2) {
        fret = Note.GuitarFret.Red;
    } else if (inputMask == 4) {
        fret = Note.GuitarFret.Yellow;
    } else if (inputMask == 8) {
        fret = Note.GuitarFret.Blue;
    } else if (inputMask == 16) {
        fret = Note.GuitarFret.Orange;
    }

    This allowed me to get something working without delving into complexities.

    I could now create a valid note object and add it to the chart.

    // create note
    var note = new Note(u_roundedTick, fret);
    
    // add note to chart
    List<SongObject> currentlyAddingNotes = new List<SongObject>();
    currentlyAddingNotes.Add(note);
    
    editor.commandStack.Push(new SongEditAdd(currentlyAddingNotes));

    Time to test it out.

    I built Moonscraper, booted up Gameplay Mode and started strumming out notes. I paused, scrubbed back and saw a solitary Green note on the chart. Success!

    First note written successfully

    But there was only one Green note on the very first line of the highway. Where were the rest of the notes? I went back into Visual Studio to investigate and realised the Note object needed instantiating with a tick, not a timestamp.

    A timestamp represents milliseconds, whereas a tick is the discrete integer representing the position on the chart, you can’t place more notes on a chart than there are tick positions to place them in. Depending on the songs BPM there would be hundreds of ticks a second.

    Time to Ctrl+F again, this time digging up the TimeToTick method of the Song object. Conveniently the Editor had access to the CurrentSong which allowed me to easily convert the time in milliseconds to the tick location.

    var tick = currentSong.TimeToTick(time, currentSong.resolution);

    So we had our fret object, the ability to add this to the editor, and the tick at which to place it in the chart. We were done.

    Step 5 - Improving the code

    When strumming this time I could see the notes appearing underneath the strikeline. I stopped playing, scrolled back in excitement and saw the notes I had just strummed persisting in the chart.

    Multiple notes

    I was ready to call it a day. But then I realized these notes weren’t aligned with the chart. Occasionally a note would line up, but most of them were off tempo and it was a mess.

    Not quite my tempo

    Then I remembered about the conversion from timestamps to ticks. There were hundreds of ticks per second so most strums wouldn’t land on major lines of the chart. I needed a rounding function.

    I wrote out the timestamps and the corresponding ticks.

    Timestamp to tick

    For the song I was currently charting I wanted notes rounded to the nearest 16th. With a bit of maths, some messy code and some trial and error, I built a rounding function:

    var tick = currentSong.TimeToTick(time, resolution);
    int strummedTick = (int)tick;
    
    int multiplyFactor = 4;
    int step = 16;
    
    float denominator = multiplyFactor / (float)step;
    int factor = (int)(resolution * denominator);
    
    int roundedPosition = strummedTick / factor;
    int roundedTick = roundedPosition * factor;
    uint u_roundedTick = (uint)roundedTick;

    The first instance of this was hardcoded to round to the nearest 16th note, but I realised the step value within Moonscraper could be used for rounding.

    Tick rounding

    This simple change below allowed the charter to increase the step when charting sections with faster notes and reduce the step for slower sections to reduce the chance notes were rounded to the incorrect line. Significantly improved usability at zero coding cost, now that is rare.

    //int step = 16; // snap notes to the step selected
    int step = GameSettings.step; // snap notes to the step selected

    With this simple change I built Moonscraper again.

    Multiple notes correctly

    This time the notes were placed evenly along the highway, rounding to the selected step size in the UI.

    6 hours in and I had made my idea reality. Live charting was now possible in Moonscraper!

    Project complete

    I was very happy with myself. These projects can often spiral out of control with increasing scope, so completing something like this in an afternoon wasn’t too shabby and it was very satisfying to complete.

    What used to be a painful effort in going back and forth in a continuous loop of play, pause, place note and rewind had now been replaced by a plug and play approach, allowing you to chart on the go in real-time with the music.

    Charting before Charting after

    Sharing with the creator

    Feeling rather pleased with myself, I contacted the creator of Moonscraper to share my incredible contribution (joking) and get it merged into the main repo.

    Sharing with the creator

    I’m sure he’ll love it.

    Profit?!

    Well not exactly..

    Feedback from the creator

    The feature had already been considered and hadn’t been built due to concerns of lowering chart standards by making it easier to create charts. It was also require creating a new mode to prevent playing and editing modes clashing with each other.

    But I wasn’t too disappointed. I was actually kind of glad I hadn’t known ahead of time or I might’ve not bothered with the project in the first place. While the maintainer didn’t want to accept a PR, I still learnt a lot and enjoyed the process. The change itself isn’t even too useful for songs with standard note placement and repeating patterns. But for laying the foundations of songs with irregular non-repeating patterns, such as Sail on Soothsayer by Buckethead, I find myself using live charting and being glad that I made the idea a reality.

    Wrap up

    This completes my write up of this side project. I hope you enjoyed reading, and if you’re interested in seeing the full code, here’s my commit on GitHub. A big thanks to FireFox2000000 for creating Moonscraper and allowing me to include our Discord conversation above.


    Enjoyed this post? ❤️

    Subscribe by email or RSS.



    Contact