Calling SOAP APIs from Rust

Approximately 15 years ago, I was running the engineering team at AppAssure Software, working on what was at the time a cutting-edge new feature: the ability to restore our backups of your physical servers, to a VMWare virtual machine! This required solving many challenging problems, not least being figuring out the ways in which your Windows install had to be manipulated to ensure it would properly boot into a guest VM when it had spent its life up to that point inhabiting a physical Dell or HP or Supermicro server. However, such low-level technical challenges paled in comparison to the horror of integrating with the VMWare vSphere SOAP API!

I can’t remember every hard thing I did over my long career, but that VMWare API integration stands out for how much of a slog it was. So you can imagine my surprise when I found myself, in 2024, working on a cutting-edge new feature for my current company (Elastio Software), and I once again needed to programmatically control a VMWare ESXi hypervisor, this time from Rust.

I figured “obviously 15 years have passed, it’ll be a nice REST API now, probably with OpenAPI definitions, this should take a day or two to get working”. As I started to dig into the VMWare documentation, I was shocked to find that the diabolical SOAP API from all those years ago is still the primary way to interact with VMWare vSphere! VMWare have since published Python and Go SDKs to wrap the worst of its many excesses, and the docs are quite a bit better now, but it’s still a SOAP API, with a WSDL definition, and all the attendant complexity that implies!

In my travels through the VMWare docs (made harder by the recent Broadcom acquisition which summarily broke all vmware.com URLs) I noticed that there is the beginnings of a REST/JSON based API, but it’s new in ESXi 8, and what I’m doing needs to work with 7.x as well, so I’m stuck with SOAP.

This led me to look around for SOAP implementations in Rust. Not surprisingly, it’s slim pickings. If you’re even a little bit younger than me you probably don’t even know what SOAP is, but even if you do, you probably never once thought to yourself “you know what, it would be fun to write the first and only SOAP client implementation in Rust!” I certainly didn’t. Why spend energy and time on something that is so obviously a dead end, something that manifests the worst of early 2000’s architecture astronaughtics and object-oriented excess?

I’m pleased to report that someone at least took a crack at it, albeit with a number of limitations:

Github user mibes wrote zeep, which conceptually is a bit like PROST, Rust’s protobuf library. You feed Prost a .proto file describing a Protobuf schema, and it generates some Rust files with structs that correspond to the PB messages, and can serialize/deserialize to/from Protobuf. Zeep does the same, except it’s input is a WSDL or XSD schema file, and it generates Rust code that can serialize/deserialize to/from the XML types that are defined therein.

Unlike Prost, Zeep does attempt to generate SOAP client code.

I was able to invoke Zeep from build.rs pretty easily, but when I fed it the insanely huge VMWare vSphere WSDL, it failed in a few unfortunate ways:

• It could not traverse multiple levels of <include> elements, which the VMWare WSDL has in abdunance
• It didn’t property keep track of the type names and aliases of the complexType types the WSDL and XSD files defined, resulting in a lot of code that would not compile
• It doesn’t have a good way to implement XSD’s very 90s Java EE style of inheritance in Rust terms, so several key polymorphic types that the vSphere API uses extensively were not propertly expressed in Rust.

I tried to work around some of these by writing some Rust code using the syn crate to parse the generated Rust and then modify it to hack around some of the flaws in the generated code, but with each horrific and depraved act I hated myself more, and only uncovered yet more flaws in the generated code.

I then thought I’d take a different approach, and use the xml-schema crate from media-io to generate just the Rust structs that correspond to the XML types in the XSD, and then write the SOAP client code myself. This, sadly, was also a failed experiment. First I manually extracted the XSD type definitions from the WSDL XML, and combined it with type definitions in the other XSD files that the WSDL file includes, into one big hairy ball of XSD that I could later on sell to a hostile regime as a bioweapon. I then tried to get xml-schema to generate Rust structs from this XSD, but ultimately it failed in similar ways.

On the plus side, as a result of my experiments with Zeep and xml-schema, I had a bunch of Rust structs that sort-of resembled the XSD types. I say “a bunch”; I am not exaggerating. The WSDL and XSD files that define the vSphere SOAP API are almost 6MB of XML. The generated Rust structs were in a single .rs file that was 9MB large!!! Neither rust-analyzer nor rustc were amused in the slightest by this abomination, and I wouldn’t say it was my proudest moment either. Compile times were miserable, my Neovim editing experience was almost as bad as mid-2000s Eclipse, I still didn’t have types I could actually use with the real SOAP API due to the aforementioned limitations.

The one advantage I had this time compared to the integration I build 15 years ago was that this time I only needed to use a tiny fraction of the API. Enough that I could probably hand-roll the structures I needed, using the ones generated by zeep as the starting point. It would be a bit tedious, but compared to forking Zeep and fixing the issues there it would be much faster (and, yes, more selfish).

Over a couple of late nights, wove an intricate tapestry of profanity as inscrutable and arcane as SOAP itself, and I also got a working SOAP client implementation going. Maybe someday we’ll open-source the code, but then someone might see it and judge me for the horrible things I had to do, but you weren’t there and you don’t know what it was like so don’t judge me!

If you are reading this because you are faced with a similar situation, I have the following advice:

• Don’t. Surely there’s a newer REST API, or a different product that has a better API, or a different profession you could take up that doesn’t involve SOAP APIs.
• Failing that, try to use zeep to generate SOAP client code and structs for your API. My sense is that Zeep only struggled in my case because of the insane surface area and complexity of the vSphere API. If zeep doesn’t panic, generates code that compiles, and you can make it work with your API, then you have dodged a bullet and should go star the zeep project on Github to show the author your gratitude for the horrors that they have saved you from.
• If zeep doesn’t work for you, try hacking around your source WSDL/XSD, commenting-out problematic structures, until it can generate something that compiles, and then manually modify the generated code to try to get the specific operations that matter to you to work right. Beware in particular XSD inheritance where a xsd:complexType extends some other type. If this inheritance is used in a polymorphic way where an operation takes the base type but does different things depending on which subtype you pass in (or returns a subtype in a similar way), you will need to roll your own YaSerialize and YaDeserialize implementation that looks at the xsi:type attribute to determine which concrete type to use.

To the authors of the zeep and xml-schema projects, I salute you for your efforts, and I hope that you are masochistic enough to continue to maintain and improve these projects, so that others may not have to suffer as I have.