diff --git a/README.md b/README.md index 3053aef..c62b51f 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,8 @@ Immediate goals: - [ ] send a surface to stand on - [ ] handle chat - NBT library - - [ ] parse NBT - - [ ] decode and encode data + - [x] parse NBT + - [x] decode and encode data ## Project Setup Instructions diff --git a/resources/README.md b/resources/README.md index 818dc15..db7964b 100644 --- a/resources/README.md +++ b/resources/README.md @@ -1,33 +1,53 @@ -# Tests +# Resources + +Data used for testing and understanding the internals of Minecraft/FOSS-VG, +also contains code to produce such data + + +## java/ + +Programs to produce data for figuring out how to interact +with Java-specific things + +- `JavaStringGenerator.java`: A simple tool that reads from stdin and outputs Java-style UTF-8 to stdout +- `netty/`: Figure out how Netty operates + + +## NBT_data/ + +Data used to test the NBT library + +`bare_int64_tag`: What the name says +`bare_int64_tag_and_int32_tag`: What the name says +`simple_nbt`: A simple NBT file containing all tag types +`nested_compounds_and_lists`: A combination of nested compound and list tags (parser stress test) +`servers.dat`: Pulled from my Minecraft installation +`servers.dat_nbt_decoded.txt`: The same data manually decoded (This was done to get a better understanding how NBT works, might come in handy.) +`level.dat`: Pulled from one of my world saves (gzip-compressed) +`level.dat_decompressed`: The same data decompressed + ## unicode_data/ -This directory contains two files with unicode data. One is in Java format, the other is normal UTF-8. +Files with unicode data +- Bare ("normal") UTF-8 +- Java format + + +## all_bytes + +Every possible 8-bit byte in ascending order ## check_endianness.cpp -A simple tool to determine the endianness of the system and write the endianness header for FOSS-VG +A simple tool to determine the endianness of the system and write +the endianness header for FOSS-VG Supports: Little Endian, Big Endian, PDP Endian, Honeywell Endian -Note that, while it can detect PDP and Honeywell endianness, the FOSS-VG project itself does not support these. +Usage: `check_endianness > header_file` -Usage example: `check_endianness > header_file` - - -## JavaStringGenerator.java - -A simple tool written in Java that takes an input as UTF-8 and outputs it in Java-style UTF-8. - -Usage example: `echo -ne "\x00" | java JavaStringGenerator > output_file` - - -## NBT_data - -Data used to test the NBT library. - -`servers.dat`: My current servers.dat as pulled from my Minecraft installation -`servers.dat_nbt_decoded.txt`: The same file manually decoded (I did this to get a better understanding how NBT works, might come in handy in the future.) -`simple_nbt`: A simple NBT file containing all tags -`nested_compounds_and_lists`: A combination of nested compound and list tags intended to be challenging to parse +Note that, while this tool should in theory be able to detect +PDP and Honeywell-316-style endianness, the FOSS-VG project itself +does not support these. diff --git a/resources/dumpnbt_output_format b/resources/dumpnbt_output_format deleted file mode 100644 index bfac304..0000000 --- a/resources/dumpnbt_output_format +++ /dev/null @@ -1,354 +0,0 @@ -############################################################################# -# Examples for all types of tags: # -############################################################################# - -[0: End]: -|–Header: 1 byte -|–Payload: 0 bytes -'–Total: 1 byte - -[0: 8 Bit Integer] name: -|–Header: 7 bytes -|–Payload: 1 byte -|–Total: 8 bytes -'–Value: 0 - -[0: 16 Bit Integer] name: -|–Header: 7 bytes -|–Payload: 2 bytes -|–Total: 9 bytes -'–Value: 0 - -[0: 32 Bit Integer] name: -|–Header: 7 bytes -|–Payload: 4 bytes -|–Total: 11 bytes -'–Value: 0 - -[0: 64 Bit Integer] name: -|–Header: 7 bytes -|–Payload: 8 bytes -|–Total: 15 bytes -'–Value: 0 - -[0: Float] name: -|–Header: 7 bytes -|–Payload: 4 bytes -|–Total: 11 bytes -'–Value: 0 - -[0: Double] name: -|–Header: 7 bytes -|–Payload: 8 bytes -|–Total: 15 bytes -'–Value: 0 - -[0: Array of 8 Bit Integers] name: -|–Header: 7 bytes -|–Payload: 7 bytes -|–Total: 14 bytes -|–Length: 3 -'–Values: - |–0 - |–0 - '–0 - -[0: String] name: -|–Header: 7 bytes -|–Payload: 14 bytes -|–Total: 21 bytes -'–Value: Hello World! - -[0: List] name: -|–Header: 7 bytes -|–Payload: 8 bytes -|–Total: 15 bytes -|–Contained Type: 8 Bit Integer -|–Length: 3 -| -|–[12: 8 Bit Integer]: -| |–Payload: 1 byte -| |–Total: 1 byte -| '–Value: 0 -| -|–[13: 8 Bit Integer]: -| |–Payload: 1 byte -| |–Total: 1 byte -| '–Value: 0 -| -'–[14: 8 Bit Integer]: - |–Payload: 1 byte - |–Total: 1 byte - '–Value: 0 - -[0: Compound] name: -|–Header: 7 bytes -|–Payload: 25 bytes -|–Total: 32 bytes -|–Length: 4 -| -|–[7: 8 Bit Integer] name: -| |–Header: 7 bytes -| |–Payload: 1 byte -| |–Total: 8 bytes -| '–Value: 0 -| -|–[15: 8 Bit Integer] name: -| |–Header: 7 bytes -| |–Payload: 1 byte -| |–Total: 8 bytes -| '–Value: 0 -| -|–[23: 8 Bit Integer] name: -| |–Header: 7 bytes -| |–Payload: 1 byte -| |–Total: 8 bytes -| '–Value: 0 -| -'–[31: End]: - |–Header: 1 byte - |–Payload: 0 bytes - '–Total: 1 byte - -[0: Array of 32 Bit Integers] name: -|–Header: 7 bytes -|–Payload: 16 bytes -|–Total: 23 bytes -|–Length: 3 -'–Values: - |–0 - |–0 - '–0 - -[0: Array of 64 Bit Integers] name: -|–Header: 7 bytes -|–Payload: 28 bytes -|–Total: 35 bytes -|–Length: 3 -'–Values: - |–0 - |–0 - '–0 - -############################################################################# -# Output for simple_nbt (what it's supposed to look like) # -############################################################################# - -[0: Compound]: -|–Header: 3 bytes -|–Payload: 475 bytes -|–Total: 478 bytes -|–Length: 15 -| -|–[3: String] Spaces and special characters are allowed in tag names, right?: -| |–Header: 65 bytes -| |–Payload: 24 bytes -| |–Total: 89 bytes -| '–Value: Idk. Let’s find out. -| -|–[92: Compound] compound: -| |–Header: 11 bytes -| |–Payload: 45 bytes -| |–Total: 56 bytes -| |–Length: 3 -| | -| |–[103: 32 Bit Integer] some_number: -| | |–Header: 14 bytes -| | |–Payload: 4 bytes -| | |–Total: 18 bytes -| | '–Value: -754506943 -| | -| |–[121: String] some_text: -| | |–Header: 12 bytes -| | |–Payload: 14 bytes -| | |–Total: 26 bytes -| | '–Value: eat a cookie -| | -| '–[147: End]: -| |–Header: 1 byte -| |–Payload: 0 bytes -| '–Total: 1 byte -| -|–[148: Double] double: -| |–Header: 9 bytes -| |–Payload: 8 bytes -| |–Total: 17 bytes -| '–Value: 623593.6542742235 -| -|–[165: Float] float: -| |–Header: 8 bytes -| |–Payload: 4 bytes -| |–Total: 12 bytes -| '–Value: 35.2678337097168 -| -|–[177: 16 Bit Integer] int16: -| |–Header: 8 bytes -| |–Payload: 2 bytes -| |–Total: 10 bytes -| '–Value: 2000 -| -|–[187: 32 Bit Integer] int32: -| |–Header: 8 bytes -| |–Payload: 4 bytes -| |–Total: 12 bytes -| '–Value: 10101010 -| -|–[199: Array of 32 Bit Integers] int32_array: -| |–Header: 14 bytes -| |–Payload: 20 bytes -| |–Total: 34 bytes -| |–Length: 4 -| '–Values: -| |–398452796 -| |–43259 -| |–2147483647 -| '–1634890337 -| -|–[233: 64 Bit Integer] int64: -| |–Header: 8 bytes -| |–Payload: 8 bytes -| |–Total: 16 bytes -| '–Value: 810001800766 -| -|–[249: Array of 64 Bit Integers] int64_array: -| |–Header: 14 bytes -| |–Payload: 44 bytes -| |–Total: 58 bytes -| |–Length: 5 -| '–Values: -| |–239865 -| |–23586749 -| |–9223372036854775807 -| |–188944201329624 -| '–3116157694992754 -| -|–[307: 8 Bit Integer] int8: -| |–Header: 7 bytes -| |–Payload: 1 byte -| |–Total: 8 bytes -| '–Value: 100 -| -|–[315: Array of 8 Bit Integers] int8_array: -| |–Header: 13 bytes -| |–Payload: 12 bytes -| |–Total: 25 bytes -| |–Length: 8 -| '–Values: -| |–113 -| |–53 -| |–119 -| |–98 -| |–84 -| |–100 -| |–245 -| '–50 -| -|–[340: List] list_int8: -| |–Header: 12 bytes -| |–Payload: 10 bytes -| |–Total: 22 bytes -| |–Contained Type: 8 Bit Integer -| |–Length: 5 -| | -| |–[357: 8 Bit Integer]: -| | |–Payload: 1 byte -| | |–Total: 1 byte -| | '–Value: 65 -| | -| |–[358: 8 Bit Integer]: -| | |–Payload: 1 byte -| | |–Total: 1 byte -| | '–Value: 96 -| | -| |–[359: 8 Bit Integer]: -| | |–Payload: 1 byte -| | |–Total: 1 byte -| | '–Value: 78 -| | -| |–[360: 8 Bit Integer]: -| | |–Payload: 1 byte -| | |–Total: 1 byte -| | '–Value: 127 -| | -| '–[361: 8 Bit Integer]: -| |–Payload: 1 byte -| |–Total: 1 byte -| '–Value: -6 -| -|–[362: List] list_strings: -| |–Header: 15 bytes -| |–Payload: 77 bytes -| |–Total: 92 bytes -| |–Contained Type: String -| |–Length: 12 -| | -| |–[382: String]: -| | |–Payload: 8 bytes -| | |–Total: 8 bytes -| | '–Value: Pacman -| | -| |–[390: String]: -| | |–Payload: 5 bytes -| | |–Total: 5 bytes -| | '–Value: ate -| | -| |–[395: String]: -| | |–Payload: 5 bytes -| | |–Total: 5 bytes -| | '–Value: all -| | -| |–[400: String]: -| | |–Payload: 5 bytes -| | |–Total: 5 bytes -| | '–Value: the -| | -| |–[405: String]: -| | |–Payload: 6 bytes -| | |–Total: 6 bytes -| | '–Value: dots -| | -| |–[411: String]: -| | |–Payload: 4 bytes -| | |–Total: 4 bytes -| | '–Value: so -| | -| |–[415: String]: -| | |–Payload: 5 bytes -| | |–Total: 5 bytes -| | '–Value: now -| | -| |–[420: String]: -| | |–Payload: 8 bytes -| | |–Total: 8 bytes -| | '–Value: he’s -| | -| |–[428: String]: -| | |–Payload: 8 bytes -| | |–Total: 8 bytes -| | '–Value: coming -| | -| |–[436: String]: -| | |–Payload: 5 bytes -| | |–Total: 5 bytes -| | '–Value: for -| | -| |–[441: String]: -| | |–Payload: 5 bytes -| | |–Total: 5 bytes -| | '–Value: the -| | -| '–[446: String]: -| |–Payload: 8 bytes -| |–Total: 8 bytes -| '–Value: words. -| -|–[454: String] string: -| |–Header: 9 bytes -| |–Payload: 14 bytes -| |–Total: 23 bytes -| '–Value: Hello World! -| -'–[477: End]: - |–Header: 1 byte - |–Payload: 0 bytes - '–Total: 1 byte diff --git a/resources/JavaStringGenerator.java b/resources/java/JavaStringGenerator.java similarity index 100% rename from resources/JavaStringGenerator.java rename to resources/java/JavaStringGenerator.java