Neon: Node + Rust

I bring to your attention the translation of the article " Neon: Node + Rust ".

Javascript for programmers who were intrigued by the rusty topic of fearless programming (making system [low-level] programming safe and fun), but waiting for inspirations or magic pendals - I have them! I worked here a little on Neon - a set of APIs and tools that make it really easy to write native extensions under Node on Rust.

TL; DR:

• Neon is an API for creating fast, reliable native Node extensions on Rust.
• Neon allows Rust's parallelism with guaranteed thread safety
• Neon-cli makes it easy and easy to create a Neon project and gives an easy start ... and finally ...
• project needs help!

I learned to cook Rust, you learn too

I wanted to make the process as easy as possible ( Larry Wall started with this too, approx. Translator ) and wrote for this Neon-cli , a console utility that generates the Neon template of the project into one command, which is assembled as usual npm install
It's all very simple. In order to build our first module with Neon, set Neon-cli: npm install -g neon-cli , then create, build and run:
')

 % neon new hello ...follow prompts... % cd hello % npm install % node -e 'require("./")' 

For particularly unbelievers, I posted a screencast here, so you can go and see.

I catch you on the word [Take Thee at thy Word]

To demonstrate the capabilities of Neon, I created a small demo (counts the number of words). The demo is simple - we read the complete collection of Shakespeare's plays and count the number of occurrences of the word “you” ( I Take Thee at thy Word - a quote from Romeo and Juliet ) At first I tried to do it on vanilla javascript. First, my code breaks the text into lines and counts the number of entries found for each line:

 function search(corpus, search) { var ls = lines(corpus); var total = 0; for (var i = 0, n = ls.length; i < n; i++) { total += wcLine(ls[i], search); } return total; } 

Search in the string includes word splitting and comparing each word with the search term:

 function wcLine(line, search) { var words = line.split(' '); var total = 0; for (var i = 0, n = words.length; i < n; i++) { if (matches(words[i], search)) { total++; } } return total; } 

The details left behind the frame can be seen in this code , it is small and autonomous (without dependencies)
On my laptop, the code works through all the plays of Shakespeare for 280-290ms. Not so bad, but as they say, there is something to strive for.

And to rural fun, we betray [Fall Into our Rustic Revelry]

One of the most remarkable features of Rust is that extremely effective code can be surprisingly compact and readable. In the Rust version, the string counting code for strings looks almost like JS code:

 let mut total = 0; for word in line.split(' ') { if matches(word, search) { total += 1; } } total //  Rust   `return`    

In fact, such code can be written with higher-level abstractions without loss of performance using iterative (iterating) methods like filter and fold (analogs of Array.prototype.filter and Array.prototype.reduce in JS):

 line.split(' ') .filter(|word| matches(word, search)) .fold(0, |sum, _| sum + 1) 

My experiments (in haste) showed even a slight (for a couple of milliseconds) performance gains. I think this is a great demonstration of the Rust-based paradigm of zero-cost abstractions, where high-level abstractions end up with comparable or even superior in performance (due to additional features for optimization, for example, the rejection of border checks) than low-level and more complicated code.
On my typewriter Rust-ovsky version works for 80-85ms. Not bad, a threefold increase only due to the use of Rust-a, and with approximately the same amount of code (60 lines in JS, 70 - Rust). And by the way, I strongly round the numbers here - this is not a rocket scince, I just want to show that you can get a significant performance boost using Rust, but it all depends on the situation.

And the thread of their life is spun [Their Thread of Life is Spun]

But that is not all! Rust allows us to do something more interesting for Node: we can easily and easily parallelize our code, and without nightmares and cold sweat caused by multithreading . Let's take a look at the implementation of this on Rust:

 let total = vm::lock(buffer, |data| { let corpus = data.as_str().unwrap(); let lines = lines(corpus); lines.into_iter() .map(|line| wc_line(line, search)) .fold(0, |sum, line| sum + line) }); 

vm::lock API gives Rust threads secure access to the Buffer Node object (that is, to a strongly typed array), while blocking the execution of the JS code.

To demonstrate how easy it was, I used the new Rayon from Niko Matsakis - a set of beautiful abstractions for parallel data processing. The changes in the code are minimal - just change the chain into_iter/map/fold/ to this:

 lines.into_par_iter() .map(|line| wc_line(line, search)) .sum() 

Please note - Rayon was not designed specifically for Neon, just Rayon implements the Rust iterator protocol, so Neon can use it out of the box.
With these minor changes, my dual-core MacBook Air runs the demo in 50ms instead of 85ms in the previous version.

Bridge Most Valiantly, with Excellent Discipline

I tried to make the integration as smooth as possible. From the Rust side, the Neon functions follow a simple protocol, receive a Call object, and return a JavaScript value:

 fn search(call: Call) -> JS<Integer> { let scope = call.scope; // ... Ok(Integer::new(scope, total)) } 

The scope object safely tracks handles in the V8 heap. Neon API uses the Rust type system - it gives a guarantee that your module will not drop the application by improperly managing Handles objects ( they pick Node guts here, use Handle at the same time, you can see ... 2009, now they don’t write on Habré anymore ... ) .

From the JS side, loading the module is simple to ugliness:

 var myNeonModule = require('neon-bridge').load(); 

Why this noise? [Wherefore is this noise]

I hope this demo will be enough to interest you. In addition to the fan, I think speed and parallelism are strong arguments for using Rust in Node. As the Rust ecosystem grows, it can be a good opportunity to get Node access to the Rust libs. As a result, I hope Neon can become a good level of abstraction which will make the process of writing extensions for Node less painful. With projects like node-uwp, it may even be worth exploring the development of Neon towards the level of abstraction over the JS-engine ( whatever that means ).

In general, there is a lot of opportunities, but ... I need help ! For those who want to participate - I created a chat in Slack for the community , you can get an invite here , as well as the #neon IRC channel on Mozilla IRC ( irc.mozilla.org ).

thanks

There is a lot more to understand and tons of unfinished work, but even what was done would be impossible without help: Andrew Oppenlander's blog post made me feel the ground, Ben Noordhuis and Marcin Cieślak taught V8 to cook, I dragged off a couple of tricks from the villainally brilliant code written Nathan Rajlich; Adam Klein and Fedor Indutny helped me understand the V8 API, Alex Crichton helped me with the mystery of compilation and linking, Niko Matsakis helped with the design of the API secure memory management, and Yehuda Katz helped with the rest of the design.

If you at least understand something from what has been said - perhaps you can help too !