Skip to content

Commit

Permalink
Added support for rebuilding modified streams.
Browse files Browse the repository at this point in the history
  • Loading branch information
@NickstaDB
NickstaDB committed Dec 19, 2018
1 parent 0a0d03f commit 36aacfa
Show file tree
Hide file tree
Showing 2 changed files with 174 additions and 10 deletions.
71 changes: 63 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
# SerializationDumper
A tool to dump Java serialization streams in a more human readable form.
A tool to dump and rebuild Java serialization streams and Java RMI packet contents in a more human readable form.

The tool does not deserialize the stream (i.e. objects in the stream are not instantiated), so it does not require access to the classes that were used in the stream*.

This tool was developed to support research into Java deserialization vulnerabilities after spending many hours manually decoding raw serialization streams to debug code!

Download v1.01 built and ready to run from here: [https://github.com/NickstaDB/SerializationDumper/releases/download/v1.01/SerializationDumper-v1.01.jar](https://github.com/NickstaDB/SerializationDumper/releases/download/v1.01/SerializationDumper-v1.01.jar "SerializationDumper-v1.01.jar")
Download v1.1 built and ready to run from here: [https://github.com/NickstaDB/SerializationDumper/releases/download/1.1/SerializationDumper-v1.1.jar](https://github.com/NickstaDB/SerializationDumper/releases/download/1.1/SerializationDumper-v1.1.jar "SerializationDumper-v1.1.jar")

\* See the limitations section below for more details.

**Update 19/12/2018:** SerializationDumper now supports rebuilding serialization streams so you can dump a Java serialization stream to a text file, modify the hex or string values, then convert the text file back into a binary serialization stream. See the section below on [Rebuilding Serialization Streams](#rebuilding-serialization-streams) for an example of this.

## Building
Run `build.sh` or `build.bat` to compile the JAR from the latest sources.

## Usage
SerializationDumper can take input in the form of hex-ascii encoded bytes on the command line, hex-ascii encoded bytes in a file, or a file containing raw serialized data. The following examples demonstrate its use:

$ java -jar SerializationDumper-v1.0.jar ACED000574000441424344707100fe0000
$ java -jar SerializationDumper-v1.1.jar aced0005740004414243447071007e0000
STREAM_MAGIC - 0xac ed
STREAM_VERSION - 0x00 05
Contents
Expand All @@ -25,15 +27,15 @@ SerializationDumper can take input in the form of hex-ascii encoded bytes on the
Value - ABCD - 0x41424344
TC_NULL - 0x70
TC_REFERENCE - 0x71
Handle - 16646144 - 0x00 fe 00 00
Handle - 8257536 - 0x00 7e 00 00

$ java -jar SerializationDumper-v1.0.jar -f hex-ascii-input-file.txt
$ java -jar SerializationDumper-v1.1.jar -f hex-ascii-input-file.txt
STREAM_MAGIC - 0xac ed
STREAM_VERSION - 0x00 05
Contents
TC_NULL - 0x70

$ java -jar SerializationDumper-v1.0.jar -r raw-input-file.bin
$ java -jar SerializationDumper-v1.1.jar -r raw-input-file.bin
STREAM_MAGIC - 0xac ed
STREAM_VERSION - 0x00 05
Contents
Expand All @@ -50,10 +52,63 @@ SerializationDumper can take input in the form of hex-ascii encoded bytes on the
superClassDesc
TC_NULL - 0x70

## Rebuilding Serialization Streams
As of 19/12/2018, SerializationDumper can do the reverse operation and convert a dumped serialization stream back into a raw byte stream. This can be useful for working with raw serialized streams because modifications can be made to the dumped text and be "recompiled" back into a byte stream.

### Example Usage
To demonstrate the use of the stream rebuilding functionality, let's start with a simple serialization stream.

aced0005740004414243447071007e0000

We can dump this using SerializationDumper, as shown above, to get the following:

STREAM_MAGIC - 0xac ed
STREAM_VERSION - 0x00 05
Contents
TC_STRING - 0x74
newHandle 0x00 7e 00 00
Length - 4 - 0x00 04
Value - ABCD - 0x41424344
TC_NULL - 0x70
TC_REFERENCE - 0x71
Handle - 8257536 - 0x00 7e 00 00

To modify the string value from `ABCD` to `AAAABBBB` we must update the hex-ascii values for both the string length and the string value as follows:

STREAM_MAGIC - 0xac ed
STREAM_VERSION - 0x00 05
Contents
TC_STRING - 0x74
newHandle 0x00 7e 00 00
Length - 4 - 0x00 08
Value - ABCD - 0x4141414142424242
TC_NULL - 0x70
TC_REFERENCE - 0x71
Handle - 8257536 - 0x00 7e 00 00

If we save this to the file `dumped.txt` then we can rebuild the stream as follows:

$ java -jar SerializationDumper-v1.1.jar -b dumped.txt rebuilt.bin

The file `rebuilt.bin` will now contain the raw bytes of the modified serialization stream. If we encode that file as hex-ascii we get the following:

aced000574000841414141424242427071007e0000

See the limitations section below for stream rebuilding limitations.

## Limitations
The tool currently doesn't support the full serialization specification. If you have something it can't dump please get in touch with some sample data, unless the stream contains an externalContents element.

'externalContents': If a class implements the interface java.io.Externalizable then it can use the *writeExternal* method to write custom data to the serialization stream. This data can only be parsed by the corresponding *readExternal* method and it is not possible to read the data without access to the original class. Such classes will have the SC\_EXTERNALIZABLE flag set in the classDescFlags field but they will not have the SC\_BLOCKDATA flag set.
###Deserialization/Dump Mode
The tool cannot deserialize all Java serialized data streams and may not be fully compliant with the Java serialization specification. In particular, if the stream contains an *externalContents* element then it cannot be deserialized without using the original class. If you have something that cannot be dumped which does not include an `externalContents` element then please get in touch with some sample data so I can look at producing a fix!

***externalContents:*** If a class implements the interface `java.io.Externalizable` then it can use the `writeExternal` method to write custom data to the serialization stream. This data can only be parsed by the corresponding `readExternal` method so it is not possible to read the data without access to the original class. Such classes will have the `SC_EXTERNALIZABLE` flag set in the `classDescFlags` field but they will not have the `SC_BLOCKDATA` flag set.

###Serialization/Rebuild Mode
The stream rebuild mode currently only operates on the hex-ascii encoded bytes from the dumped data. For that reason, changing the string "ABCD" to "AAAABBBB" won't have the desired effect of producing the bytes 0x4141414142424242 in the output file. A future update may improve this but for now you'll have to do your hex-ascii encoding of strings manually!

Length fields aren't updated automatically during stream rebuilding. This may be desirable or not, but if you modify a string value in a way that changes the length just be aware that you may also need to modify the length (hex-ascii) field accordingly. The same applies to arrays.

If the stream contains any `TC_REFERENCE` elements and you modify it to remove an element that includes a `newHandle` field then you may break the references in the stream unless you manually update them. Reference handles/IDs are incremental and start at `0x7e0000` so the first `newHandle` field is reference by `0x7e0000`, the second by `0x7e0001`, and so on. If the first element with a `newHandle` field is removed from the stream then any `TC_REFERENCE` elements in the stream must be modified to refer to a handle value one less than what they originally referred to.

## Bug Reports/Improvements
This tool was hacked together on the fly to support my own research but if you find the tool useful and have any bug reports or suggestions please get in touch either here or on Twitter ([@NickstaDB](http://twitter.com/NickstaDB "@NickstaDB on Twitter")).
113 changes: 111 additions & 2 deletions src/nb/deser/SerializationDumper.java
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
package nb.deser;

import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.LinkedList;
import nb.deser.support.ClassDataDesc;
Expand All @@ -22,6 +28,7 @@ public class SerializationDumper {
private String _indent; //A string representing the current indentation level for output printing
private int _handleValue; //The current handle value
private final ArrayList<ClassDataDesc> _classDataDescriptions; //Array of all class data descriptions to use with TC_REFERENCE classDesc elements
private boolean _enablePrinting; //Flag to control console printing (used when rebuilding to test a rebuilt serialization stream)

/*******************
* Converts the command line parameter to an ArrayList of bytes and sends
Expand All @@ -37,11 +44,15 @@ public static void main(String[] args) throws Exception {
byte[] fileContents;

//Validate number of command line args
if(args.length == 0 || args.length > 2) {
if(args.length == 0 || args.length > 3) {
System.out.println("Usage:");
System.out.println("\tSerializationDumper <hex-ascii-data>");
System.out.println("\tSerializationDumper -f <file-containing-hex-ascii>");
System.out.println("\tSerializationDumper -r <file-containing-raw-data>");
System.out.println("");
System.out.println("Rebuild a dumped stream:");
System.out.println("\tSerializationDumper -b <input-file> <output-file>");
return;
}

//A single argument must be a hex-ascii encoded byte string
Expand All @@ -53,6 +64,10 @@ public static void main(String[] args) throws Exception {
(Character.digit(args[0].charAt(i * 2 + 1), 16))
));
}
} else if(args.length == 3 && args[0].toLowerCase().equals("-b")) {
//Three arguments starting with -r means we're rebuilding a stream from a dump
sd.rebuildStream(args[1], args[2]);
return;
} else {
//Two arguments means the data is in a file, read the file contents
f = new File(args[1]);
Expand All @@ -75,6 +90,9 @@ public static void main(String[] args) throws Exception {
for(int i = 0; i < fileContents.length; ++i) {
sd._data.add(fileContents[i]);
}
} else if(args[0].toLowerCase().equals("-b")) {
System.out.println("Error: The -b option requires two parameters, the input file and the output file.");
return;
}
}

Expand All @@ -91,6 +109,7 @@ public SerializationDumper() {
this._indent = "";
this._handleValue = 0x7e0000;
this._classDataDescriptions = new ArrayList<ClassDataDesc>();
this._enablePrinting = true;
}

/*******************
Expand All @@ -99,7 +118,9 @@ public SerializationDumper() {
* @param s The string to print out.
******************/
private void print(String s) {
System.out.println(this._indent + s);
if(this._enablePrinting == true) {
System.out.println(this._indent + s);
}
}

/*******************
Expand Down Expand Up @@ -128,6 +149,20 @@ private String byteToHex(byte b) {
return String.format("%02x", b);
}

/*******************
* Convert a hex-ascii string to a byte array.
*
* @param h The hex-ascii string to convert to bytes.
* @return A byte array containing the values from the hex-ascii string.
******************/
private byte[] hexStrToBytes(String h) {
byte[] outBytes = new byte[h.length() / 2];
for(int i = 0; i < outBytes.length; ++i) {
outBytes[i] = (byte)((Character.digit(h.charAt(i * 2), 16) << 4) + Character.digit(h.charAt((i * 2) + 1), 16));
}
return outBytes;
}

/*******************
* Convert an int to a hex-ascii string.
*
Expand Down Expand Up @@ -155,6 +190,80 @@ private int newHandle() {
return handleValue;
}

/*******************
* Parse output from parseStream() and turn it back into a binary
* serialization stream.
******************/
private void rebuildStream(String dumpFile, String outputFile) {
String inputLine;
BufferedReader reader;
ByteArrayOutputStream byteStream;
FileOutputStream outputFileStream;
byte[] rebuiltStream;

//Parse the input file (a serialization stream dumped by this program)
System.out.println("Rebuilding serialization stream from dump: " + dumpFile);
try {
byteStream = new ByteArrayOutputStream();
reader = new BufferedReader(new FileReader(dumpFile));
while((inputLine = reader.readLine()) != null) {
if(inputLine.trim().startsWith("newHandle ") == false) {
if(inputLine.contains("0x")) {
if(inputLine.trim().startsWith("Value - ")) {
inputLine = inputLine.substring(inputLine.lastIndexOf("0x") + 2);
} else {
inputLine = inputLine.split("0x", 2)[1];
}
if(inputLine.contains(" - ")) {
inputLine = inputLine.split("-", 2)[0];
}
inputLine = inputLine.replace(" ", "");
byteStream.write(hexStrToBytes(inputLine));
}
}
}
reader.close();
} catch(FileNotFoundException fnfe) {
System.out.println("Error: input file not found (" + dumpFile + ").");
System.out.println(fnfe.getMessage());
return;
} catch(IOException ioe) {
System.out.println("Error: an exception occurred whilst reading the input file (" + dumpFile + ").");
System.out.println(ioe.getMessage());
return;
}

//Test the rebuilt serialization stream
System.out.println("Stream rebuilt, attempting to parse...");
this._enablePrinting = false;
rebuiltStream = byteStream.toByteArray();
for(byte b: rebuiltStream) {
this._data.add(b);
}
try {
this.parseStream();
} catch(Exception e) {
System.out.println("Warning: An exception was thrown whilst attempting to parse the rebuilt stream.");
System.out.println(e.getMessage());
}

//Save the stream to disk
try {
outputFileStream = new FileOutputStream(outputFile);
outputFileStream.write(rebuiltStream);
outputFileStream.close();
} catch(FileNotFoundException fnfe) {
System.out.println("Error: exception opening output file (" + outputFile + ").");
System.out.println(fnfe.getMessage());
return;
} catch(IOException ioe) {
System.out.println("Error: an exception occurred whilst writing the output file (" + outputFile + ").");
System.out.println(ioe.getMessage());
return;
}
System.out.println("Done, rebuilt stream written to " + outputFile);
}

/*******************
* Parse the given serialization stream and dump the details out as text.
******************/
Expand Down

0 comments on commit 36aacfa

Please sign in to comment.