#arkscript June update!

I got nerdsnipped into optimizing ArkScript by Reddit, and took it probably way to far...

https://lexp.lt/posts/arkscript_update_june_2025/

Better performances, builtins renaming to be able to import them, and better error messages!

#pldev #compiler #blog

I published the first ArkScript v4 pre-release about 2.5 years ago

Time does fly, and we're finally near this goal I set for myself years ago:

a usable language, performant and expressive enough for me to use as a #Python replacement

#ArkScript

Of course it’s not that good for performance right now, but I attribute that to having to swap the stack, I throw in a not perfect implementation and removed the old automatic state push, since it’s now the job of a dedicated instruction

I couldn’t have kept the automatic state push even with swapping arguments, because I would have still needed to move the arguments of function calls around, since the state has to be pushed before the arguments…

Ackerman 3 7 went from 33ms to 35ms, I hope to be able to go below 30ms once I invert the argument order and remove the stack swap

#ArkScript

Took me about 2 days to create an instruction that pushes the vm state to the stack before calling a function, so that we can return to the call site to resume execution of the bytecode

All of that just to remove the call stack inversion, because we push (a,b,c,…) and the functions load (a,b,c,…) too! (Which can’t work without stack inversion because we would pull a into c, b in b, c in a…)

And I want to remove stack inversion because it’s useless if I invert the loading of arguments in functions (or the push order)

One day I’ll be satisfied

#ArkScript

I asked on Reddit how I could make #ArkScript more professional, perhaps even gather a few more users

One answer was « I found benchmarks on ArkScript's main page and it doesn't look good man. it gets beaten even by Python, not to mention Lua my beloved which is the main embedded scripting language out there. maybe work on performance, […] maybe you'll catch up to Lua one day »

Well now it beats Python and Ruby on some benchmarks, I guess that’s good enough?

https://arkscript-lang.dev/benchmarks.html

Benchmarks go vroom vroom

https://codspeed.io/ArkScript-lang/Ark

#ArkScript

ArkScript is faster (double digits in % before/after) thanks to many new super instructions

And now I wonder why I didn’t do so sooner

Also small victory but on the Ackerman benchmark #ArkScript is faster than python and ruby

I ended up adding new super instructions (merging IR entities into one), first by following my own intuition to optimize something done quite frequently:

(while (< index 100) …)

This would generate 4 instructions for the condition, load symbol, load const, LT, jump if false, I compacted it to 2, load sym and LT_CONST_JUMP_IF_FALSE to do the comparison with a constant. This gave a 30% perf improvement on the comparison/looping!

Well according to my IR entities frequencies it wasn’t the most common set of entities so I have more to add, and some are… disturbing

#ArkScript #pldev #compiler #optimization

Received a new sticker sheet for #ArkScript: 100 more stickers!

Currently wondering if I should add type inference to #ArkScript, partly because I want to have fun with it and learn how it’s done, partly because it could help remove runtime checks (and maybe enable more optimizations like using typed instructions)

But I also know the biggest optimization would be inlining done in the IR to byte code compiler, though that means rewriting my IR and AST lowerer again

The partial functions are created by the arrow macro, by replacing _ with the argument / previous function call, using another macro

It helped me find a bug in the macro argument unification, regarding spreads (…args means « any value here (0 or more) stored as a list inside args », and using …args means « expand the list args and pass each value as a separate argument to the macro »), spread were resolved everywhere and checked thoroughly, generating false « can’t unify symbol to spread » errors

#ArkScript

#ArkScript has a better arrow macro `->` now!

It used to take only function names:
(-> value f1 f2 f3)
To produce (f3 (f2 (f1 value)))

Now it can take partial functions too:
(->
"data.csv"
(string:split _ "\n")
tail
(map _ print))

Would print each line of a csv file after dropping the header

#ArkScript April 2025 update is up!

https://lexp.lt/posts/arkscript_update_april_2025/

#pldev #langdev #compiler #blog

We're in may and I haven't posted a new #ArkScript update article yet, but you can read this one instead:

https://lexp.lt/posts/inst_source_tracking_in_arkscript/

I talk about how I added source tracking on a per instruction basis inside ArkScript VM and it was quite well received on the not-very-orange-website (https://www.reddit.com/r/ProgrammingLanguages/comments/1kcef2l/instruction_source_location_tracking_in_arkscript/)

#pldev #langdev #cpp

On another note, I’ve added instruction source location tracking to #ArkScript!

Meaning, we can (finally) have runtime errors that point to the line which threw the error. As well as go up the call tree and display it with the line of each call as well!

However I’m still dueling with #msvc that loves generating weird errors at runtime (and my favorite OS, Windows, using back slashes in path instead of forward slashes…)

#pldev #compiler #cplusplus

I've decided to stop trying to make everything perfect in #ArkScript. I don't have to provide a fully-fledged HTTP client and server module.

I'm now only focusing on the client side, having deleted the server code (I couldn't figure out how to make the VM thread pool work with the HTTP server thread pool).

I've created a veeery small Python 3 echo server using the WSGIref module, that makes testing so much easier too!

Turns out I was semi-wrong in my article: I can reference a local by its index on the stack, because we now have a dedicated stack for locals, and closures now own their fields and have their own lookup algorithm

I will try and implement a LOAD_SYMBOL_BY_INDEX (currently loading by id, which means we must iterate over the entire scope to find the variable we need), since it’s a new instruction it shouldn’t break things and I’ll be able to iterate

Again, I expect a significant performance boost from this as we will finally be able to load a variable value in O(1) instead of O(n), with n being the size of the current scope
#ArkScript #pldev

I indeed found a better memory layout to store variables in ArkScript, and I got a 76% performance boost on the binary tree benchmark, and a 21% perf boost on Ackermann(3, 7)
Who knew using a contiguous storage buffer could be beneficial? 🤡

I retraced all the performance improvements I applied to #ArkScript through the last five years, with updated benchmarks, AND DAMN what a journey

https://lexp.lt/posts/optimizing_scopes_data_in_arkscript_vm/

#pldev #compiler #cplusplus

I might have found yet another (better? At least on paper) memory layout for storing #ArkScript scopes and locals

Currently I create a vector<pair<id, value>> for each scope. Quite costly in terms of copies and all

What if I had a
array<pair<id, value>, N>
And my scopes were just views:
view(start, length, min id, max id)

Since only the last scope can grow… it could work. Min and max id are there for a basic bloom filter. I just have to solve the problem for closures that have their own scope that must be kept alive, but by pushing references (value holding a ptr to value) this could be solved easily

Also because of closures, scopes are shared ptr to Scope (the class holding the vec of pair) currently. Quite costly to construct…

I will go back to my old scopes/locals management code and I think I will write an article about how it evolved, and the various performance boosts it yielded

#pldev #proglang

Today I wrote 30 more solutions for #ArkScript on Rosetta Code!

https://rosettacode.org/wiki/Category:ArkScript

Adding to the wiki is a thing, the other is using those examples as tests in the project implementation