Maybe it worth to you check Remixed PD mods mechanics? It very far from complete, completely lack documentations, full of bugs, and so on. But at this point it allow do following (without touching java code):

• Replace any sprite
• Replace any text for any language already in
• Assign file & index for item sprites
• Change animation sequences for mobs ( not all special effects implemented yet )
• Change dungeon structure ( levels graph, kind, size )
• Assign custom level music
• Assign custom music for boss fight
• Add predesigned levels to dungeon ( you can use Tiled to define floor layout, place mobs and items)
• Add random levels ( this feature somewhat similar to YourPD custom maps but no gui avaliable yet )
• Create custom bestiary ( mob|level probability )
• Add signs and codexes with arbitary texts

If you interested just check assets folder in source ( https://github.com/NYRDS/pixel-dungeon-remix/tree/master/PixelDungeon/src/main/assets ) and for example ( https://github.com/NYRDS/PD_mini-RPD-MOD )

• Maze
• The Fallen

Since mod can drastically change game difficulty one cannot earn badges while playing in mod.

Tiled - map editor