Support for HHVM

Summary

This project extends ​SWIG to add a new module: the support for ​HHVM. The aim is to allow the calling of native functions (that were written in C or C++) by languages supported by HHVM, i.e. ​PHP and ​Hack. This project doesn’t support any Hack-exclusive syntax and generates common extensions that can be run from both PHP and Hack. The extension, however, is generated in Hack. It uses a native-extension API provided by HHVM, called ​HHVM-Native Interface (HNI).

The end goal from this project is to add HHVM bindings to Xapian.

SWIG

The Simplified Wrapper and Interface Generator (SWIG) is a tool used to extend native code written in C/C++ to scripting languages. The programmer writes an interface file which lists the set of classes and functions to be extended to the target interpreter/compiler. SWIG compiles this interface file to generate extensions/shared libraries that glue the native code to the target language.

HHVM

HipHop Virtual Machine (HHVM) is a virtual machine based on just-in-time (JIT) compilation approach that serves as an execution engine for the PHP and Hack programming languages. It compiles the executed PHP/Hack source code to an intermediate bytecode, which is then optimized and run as machine code.

Work done in this project

This project wraps many C/C++ constructs including functions, global variables, constants, enumerations, pointers, structures, classes, inheritance, arrays, overloaded functions and virtual/abstract classes. This should be enough to extend a typical C/C++ library. Several parts of the project have also been rewritten over time to add optimizations supported by HHVM. The project also contains a number of examples demonstrating the usage of the module.

See Journal for a weekly timeline.

Code

My Repository: ​https://github.com/ng420/swig-hhvm/commits/master?author=ng420

SWIG Branch: ​https://github.com/swig/swig/commits/gsoc2016-hhvm?author=ng420

Work done during GSoC: commits up to August 23, 2016

The code from my repository has been merged to the SWIG gsoc2016-hhvm branch after review. This branch serves as a staging area, and it will be added to SWIG distribution after the module is thoroughly tested and documentation for users is added.

Work in Progress

• Documentation
• Support for more features
• Testing and fixing bugs
• Adding more Examples

Challenging Parts

As SWIG’s documentation mentions, every language is different in its own right. HHVM presents a compiler which works with PHP, which is inherently a scripting language, designed to work with interpreters. This presents interesting set of challenges, few of which are highlighted below:

• A different module: HHVM turns out to be a completely different module as compared to any existing module. Therefore, it is difficult to come up with approaches to handle the features. While Java and C# happen to be compiled-language modules, there are several aspects (overloading for example) which are supported by them but not PHP (therefore HHVM). On the other hand, the ​gradual typing supported by HHVM is different from PHP itself, which makes the handling different from PHP. HHVM also adds many optimizations because of the typing, and if PHP-like extensions are generated which use the ‘mixed’ type extensively, we would leave speed and resource optimization on the table, which in turn, kills the point of using HHVM over PHP.

• Type handling: HHVM is gradually typed, which means many variables and objects have to be typed in the code. While, it is not as challenging for trivial types, constructs like pointers pose a serious challenge. SWIG supports incomplete types, which adds to the challenge. The pointer-handling had to be re-written several times to handle different cases. The current approach is to generate a wrapper intermediate class for any unknown type.

• Overloaded functions and methods: PHP and Hack do not support overloading, and SWIG’s overloading handler is designed to handle scripting languages, which are usually dynamically typed. Several constructs had to be rewritten to support overloading, which were restructured and debugged many times over the course of the project.

• Lack of documentation: HHVM does not have any documentation for HNI, and the developers recommend looking at existing code to understand how features are handled. While the developers have been extremely helpful on IRC, it often blocked the development because of the difference in Time Zone and no support during the weekends. Same goes with SWIG, there is a lack of updated developer documentation, and the most helpful approach is to look at other modules. While Olly has been extremely helpful with regard to the constructs in SWIG, the lack of documentation on either side combined with my partial knowledge on both often led to long thinking hours and frequent re-implementations.

• Evolving HHVM: HHVM is constantly being developed and changing quickly. One needs to keep an eye on new features and deprecated ones and support them while having backward compatibility. Fixing a version of HHVM might not be the optimal solution since the HHVM user-base is also rapidly growing. It has been and still is a challenge to keep up with the evolution of HHVM combined with ever growing SWIG to keep the module updated.

Future Work

• Add more features
• Add runtime test cases
• Support for std containers
• Cross language polymorphism
• Xapian Bindings

Closing Notes

I am extremely grateful to Olly for giving me the amazing opportunity to work on this project and for introducing me to the Open-Source world. I have been childish, frustrated, trackless at times, and you have always helped me, given me motivation to work onward. Without you, I wouldn't have dared to even begin on this project. I would also like to thank ​Orvid King for taking out his precious time to help me with HNI and HHVM in general. I'm also grateful to the HHVM community, which has been eager to help me with my project at every point of time.

I have learned a lot from this project. Projects involving compilers and interpreters are often considered intimidating. This project helped me overcome the fear, and now I am eager to work more on it. It is one of the most (probably the most) challenging project I have ever worked upon. I feel I have grown as an engineer by working on this project.

Finally, I would like to thank Google for organizing such a wonderful program. It has helped a n00b like me to get introduced to the open-source community, which is known to be a community of genius and hard-working people. Without GSoC, it would have been difficult for me to begin contributing to it. Additionally, the program has helped me have one of the productive summers I ever had.