FORUM ARCHIVED

Modder Utilities

Discussion in 'Modding' started by Glazed, May 12, 2012.

  1. Glazed

    Glazed Member

    I've been working on some tools for DoD. One of them is tool that looks at a collection of XML files and figures out their overall schema. It answers questions like,
    • What elements are in this file?
    • What possible attributes can appear on these elements?
    • How may of this kind of element can appear inside its parent element?
      • E.g., Does an item ever have more than one <weapon> element?
    It answers these questions by generating a simple text file that shows the element structure, all possible attributes, and a guess at the type of data each attribute holds. (Integer, decimal, text, image path, or true/false).

    I found this tool invaluable while trying to figure out the XML file schema and turning it into a SQL database schema. I can imagine that budding modders might also find it useful. Even experienced modders can use it to help validate their mods. It's not a perfect validation tool, but there are several kinds of errors that this tool can detect:
    • If you misspell an element name even once, it will appear as a separate entry in the file, making it obvious.
    • If you misspell an attribute name, the same is true.
    • If you accidentally enter a child element more than once that is only meant to appear once, you will see a number you don't expect.
    • If you use the wrong attribute every time for a certain element you might spot this, but then again you might not. E.g., if you always use percent="N" instead of percentage="N" you may or may not spot this. However, you can use a comparison tool to compare your mod's schema to the core game schema, and
    • If you make a typo in even one attribute which causes the parser to think it's a different data type, then this should be obvious. E.g., the artifact attribute is supposed to be a 1 or a 0, but if you enter "2" by accident, or "1=", then the parser will think the field is an integer or text instead of a true/false. As long as you know what the attribute is supposed to contain you should spot this.
    If you first generate a set of files for the core game, you can use a tool like Beyond Compare to compare your mod's schema to the core game schema. This might help you find errors. Then again, it might help you find errors in the core game. :D

    Here's a partial text schema as an example. This was generated from all of the itemDB.xml files in the core game, including both expansions as of 1.11 RC4.

    Code:
    item(697) (697)
        @alchemical(75) boolean
        @artifact(89) boolean
        @craftoutput(95) boolean
        @iconfile(697) filepath(42)
        @level(281) integer
        @maxstack(23) integer
        @name(697) text(52)
        @overrideclassname(28) text(12)
        @special(41) boolean
        @type(216) integer
        armour(1) (251)
            @level(251) integer
            @randoms(28) boolean
            @type(251) text(6)
        artifact(1) (87)
            @quality(87) integer
    
    An element name appears as a name with no prefix and includes two numbers in parentheses after it. The first number indicates the maximum number of times that element was found under a single parent element. E.g., The <item> element was found a maximum 697 times under its parent element <itemDB> (which is not shown). The second number is how many times that element was found total. For the <item> element those two numbers are the same since there is only one <itemDB> element in each itemDB.xml file, so all 697 appear under the same element. You may be thinking, "This doesn't make sense! There are three separate <itemDB> elements for the three itemDB.xml files from he core game and the two expansions!" You are correct, but I merge all of the same kind of XML file together into one big document before parsing it. So it treats all <item> elements as being under a single <itemDB> element.

    Under each element and indented with a tab is a list of possible attributes for that element, each prefixed with the @ symbol. If an attribute appears here it means that it appears at least once on one of the parent elements, somewhere in one of the XML files. It does not mean that it's on all of them. Immediately after the attribute name is the count of how many times it was found. You can see, for instance, that the "name" and "iconfile" attributes appears on all 697 items, but none of the other ones are always present. After the count is my best guess on the type of data the attribute contains:
    • boolean: This is a true/false value expressed in the attribute with a 1 (true) or a 0 (false). E.g., artifact="1".
    • filepath: This is a string of text that contains a / character. I only consider an attribute a file path if every instance of that attribute contains a / character. This means that if even just one values contains no / for some reason, it will just appear as "text" instead of "filepath". Conversely, the parser won't accidentally think it's a file path just because one value contains a /. The number after it is the length of the longest value that was found.
    • text: This is an ordinary string of text like a name or description. The number after it is the length of the longest value I found.
    • integer: This is a number with no decimal. E.g., 1, 2, 3, 4, 5.
    • decimal: This is a number with a decimal. E.g., 0.25, 1.5. This is generally only used for the scaling factors for spell damage.
    It's quite possible that my parser will mislabel an integer as a boolean. I've seen this happen before when the only items in a mod that had Righteous damage (or whatever) only had one of that damage type. Since my code never saw anything higher than a 1, it thinks it's a boolean.

    After the list of attributes under an element you might find other elements. For instance, an <item> element can have an <armour> element. Each item only has 1 <armour> element, and there are 251 of them throughout all the itemDB files. Each child element can have its own list of attributes and its own child elements. Note: not all elements have attributes. E.g., <gem/> or <mushroom/>. The presence of these elements simply indicates that the item is one of these things. No attributes are present on them.

    Special Handling for Certain Files:

    For some files I didn't just parse them as-is. I first transformed the files into something simpler and flatter. Some files contain meaningless structure. These two files are spellDB.xml and skillDB.xml.

    SpellDB.xml sometimes has its <spell> elements contained with wrapper elements:
    Code:
    <mushroomSkills>
        <spell name="Spore Stash" .... />
    </mushroomSkills>
    
    These wrapper elements have no meaning, can be anything, and aren't required. So I simply extract their <spell> elements and ignore them.

    SkillDB.xml can contain two levels of wrapper elements:
    Code:
    <warriorSkills>
        <swordSkills>
            <ability name="" ..../>
        </swordSkills>
    </warriorSkills>
    
    Again, these elements have no meaning, and aren't required, so I extract all <ability> elements from them and ignore them.

    I do not do this with monsters. Monsters can have child monsters and their position has meaning.

    Requirements:

    Windows computer with .Net framework 4.0 installed. I have no idea if this will work on the Mono platform.

    Usage:

    You just drag and drop a folder onto the EXE. It will recursively look in the folder for all appropriate XML files and load them all. If you want the entire base game then drop the whole "dungeons of dreadmor" folder onto the EXE. If you want to just look at RotDG, then just drag the "expansion" folder onto it. It will not look inside zip files, so if you want to examine a mod, unzip it first into a containing folder. I'm not responsible if you do something silly like drag your entire C drive onto the exe.

    It will make a new folder as a sibling to the dropped folder that has the same name plus " schema". E.g., "dungeons of dredmor schema". Inside will be all the text files.
     

    Attached Files:

  2. And the text files that are churned out by this look pretty if you use Matlab syntax highlighting.
     
  3. Kaidelong

    Kaidelong Member

    Source? If this doesn't work on Mono I could see about helping port it.
     
  4. Glazed

    Glazed Member

    Source:
     

    Attached Files:

  5. Null

    Null Will Mod for Digglebucks

    mushroomSkills actually DOES mean something. It's what caused spore stash to give an amount that scaled to your fungal level.
     
  6. Glazed

    Glazed Member

    That seems like a bad way to control the effect of a spell. That behavior would have to be hard-coded into the game.
     
  7. Essence

    Essence Will Mod for Digglebucks

    It was.