Candle: New machine learning framework for Rust

Artificial intelligence (AI) company Hugging Face recently released a new minimalist machine learning (ML) framework for Rust called Candle. It has garnered 7.8k stars and 283 shares on GitHub.

Hugging Face has also launched a new programming tool called SafeCoder, which leverages StarCoder to allow organizations to create their own on-premises equivalent of GitHub Copilot. Earlier this year, the open source company released a JavaScript library that allows web and front-end developers to add machine learning capabilities to websites and applications.

Hugging Face is investing in developer tools to expand the reach of its 300,000 source machine learning models, explains Jeff Boudier, the startup’s head of product and development. open.

“The big picture is we’re growing the ecosystem for developers and seeing a lot of traction doing that,” Boudier told The New Stack after a $235 million fundraising that included Support from Google, Amazon, Nvidia, Salesforce, AMD, Intel, IBM and Qualcomm. “Now with the support of all these great platforms and players, we can ensure that we have support for the community, no matter what platform they use to run the learning model my machine”.

Candles, Rust ML Frame

ML frameworks are typically written in Python and supported by frameworks like PyTorch. These frameworks tend to be “very large, making instance creation on a cluster slow,” Hugging Face explains in Candle’s FAQ.

Candle is designed to support serverless inference, which is a way to run machine learning (ML) models without having to manage any infrastructure. Candle does this by allowing deployment of lightweight binaries, the FAQ explains. Binary files are executable files that contain the files and resources required for the application to run on the target environment.

Candle also allows developers to remove Python from production workloads. “Python’s overhead can severely impact performance, and the GIL is a notorious source of headaches,” the FAQ explains, referring to the Python GIL or Global Interpreter Lock. The GIL offers many benefits but prevents CPython from achieving full multi-core performance, according to cloud storage provider Backblaze, which explains it in this blog post.

There are three Candle app demos that developers should check out:

SafeCoder: Pilot companion for businesses

One of the reasons businesses aren’t flocking to Copilot is that their code can be used to train models, meaning the data is out there. It’s no surprise that organizations are in no rush to adopt that.

SafeCoder will allow that code information to remain in place while still informing the model, Boudier explains.

Customers can build their own Code LLM, refined on their proprietary code base, using open models and libraries, he said, without sharing their code with Hugging Face or any other company. any other third party.

“With SafeCoder, Hugging Face provides a containerized, hardware-accelerated Code LLM inference solution that is deployed directly by customers in their security infrastructure without the need for code entry and completion to leaving their secure IT environment,” wrote Technology Lead Boudier and Hugging Face. Philipp Schmid in a blog announced the tool on August 22.

It is based on StarCoder, an open source LLM alternative that can be used to build chatbots or AI coding assistants. He said StarCoder is trained in 80 different programming languages, including Rust.

“StarCoder is one of the best open models for doing code recommendations,” Boudier said. “Star Coder is an open, pre-trained model that has been trained on over a trillion commercially permitted open source project data tokens. That’s the training dataset that you can look at on the Hug Face hub, you can see if any of your code is in the dataset, so it’s really built with consensus intention and compliance from the beginning.”

He added that VMware is an early adopter of SafeCoder.

“I can have a solution that is specifically designed for my company and deployed within our infrastructure so that it operates in a secure environment,” Boudier said. “That’s the promise of SafeCoder.”

Loraine Lawson is a veteran technology reporter who has covered technology issues from data integration to security for 25 years. Before joining The New Stack, she served as editor of the banking technology site, Bank Automation News. The girl has…

Read more from Loraine Lawson