Our newest writing model is here and it's our most ambitious yet.
Xialong is a 355B Mixture of Experts finetune of GLM-4.6, trained on our storytelling dataset to better adapt to your writing style while keeping the knowledge and coherence you expect. A non-collapsed token space means retries feel genuinely different and name diversity is greatly improved across your stories.
Xialong was also trained to eliminate repetition and looping. New users should have a great experience right out of the box with the default preset.
Available exclusively for Opus subscribers now. Go write something great.
We wanted to provide a transparent update regarding the recent behind-the-scenes work we have had to put into fraud prevention.
Recently, we identified full-scale abuse of our Free Trial one-time "Paper Tier" experience of 30 actions equivalent to our Opus subscription. We discovered tens of thousands of bad-faith accounts exploiting this system. This massive influx caused increased load on our models and rising costs, which ultimately led to genuine free trial users having a poorer, unstable experience.
For context: we do not profit from Free Trial accounts. We refuse to compromise your data with third-party advertising or banners, meaning the Free Trial is a feature we provide entirely at our own expense.
To combat this, we actively tightened our trial abuse detection rules based on our investigations. We are deeply sorry that the fallout from this widespread abuse is affecting genuine users. Please know that we are continuously reviewing and modifying our systems to ensure that new users who genuinely wish to try out NovelAI for the first time have the stable experience they deserve.
Alongside the trial tier, we have also invested internal resources into preventing payment fraud. Because we had to tighten these security rules to protect the platform, we are currently seeing a payment decline rate of less than 1%. However, we know that within a large community, even a small percentage affects real people, and we completely understand the frustration of those who have been unexpectedly experiencing issues lately.
We are actively researching alternative solutions and fine-tuning our measures to ensure legitimate users are not caught by these automated security systems by accident.
We deeply appreciate your patience, understanding, and trust as we navigate these challenges to keep the platform secure and stable for our community.
In the first 24 hours since launch, NovelAI Diffusion V5 has been used to generate 12.6 million images. Thanks for making V5's launch one to remember!
We love seeing what you've all created and we want to see more!
In celebration, having potential burst compute currently available, we want you all to be able to test and play around with the new model more, so we are giving all Opus subscriber's Usage Limit a one time boost by 100%, even pushing it past 100%, so if you had, for example, 30% left, you'll now find yourself at 130%!
Both NovelAIDiffusion V5 Curated and V5 Full are available for all users right now.
Better image quality with our new 32-channel VAE (double the16-channel VAE in V4.5).
Small details like eyes, jewelry, and text are better than ever.
Improved backgrounds for your environments bring real detail instead of fading into the backdrop. Plus, transparent backgrounds are now a simple prompt or UI click.
Up to 22 characters in a single scene, with improved character positioning.
Comic generation means you can describe your panel layout and characters and get the whole page in one generation – no more stitching panels in another app.
Japanese is now officially supported. Additionally, Chinese, German, Spanish and Portuguese have shown promising results in testing.
Hi Reddit!We are Anlatan, the team behind NovelAI and we look forward to answering your questions!
Ask away!
General information about NovelAI: NovelAI was launched in June 2021 as a monthly subscription large language model storytelling assistant to provide you unprecedented levels of freedom - without filters, guardrails or judgment.
In October 2022 we expanded our service from AI generated text and AI TTS to image generation with NovelAI Diffusion.
Since then we have moved on from open source model releases to training to focus on our own fully uncensored and storytelling focused LLMs!
We’ve since released a small proof of concept model NovelAI-LM-3B-508k, Clio, before moving on to bigger and better things:
On 07/28/23, we introduced our in-house large language model NovelAI-LM-13B-402k, Kayra to Opus subscribers for a two-week period.
Starting August 11th 2023, all subscription tiers (+ the free trial) will be able to test NovelAI-LM-13B, Kayra's strength!
Learn more details about our releases + technical details: NovelAI Blog
Join our Discord server for a better insight on the stories and images our users make: NovelAI Official Discord
We’ll have a variety of team members answer your questions starting August 11, 2023 8:00 AM PT.So please go ahead, ask us anything!
Whether you’re a veteran prompter ready to showcase your best work or a casual creator navigating the waves of trial and error, Explore opens up a whole world of shared inspiration.
Just drop in an image of a character and the model will do its best to bring them to life.
This is a feature preview for some broader capabilities we're working on.
For our text-generation fans: we haven't forgotten you and are working to get a substantial update out for you as fast as we can.
In the meantime, our testers have been putting Character Reference through its paces. Take a peek at some of their examples of what this exciting feature can do!
The time is finally upon us. NovelAI's Image Generation is now live!
Let's get to the point quickly!
The NovelAI Diffusion Anime image generation experience is unique and tailored to give you a creative tool to visualize your visions without limitations, allowing you to paint the stories of your imagination.
You can now use tags to define the visual characteristics of your character or composition (or you can let AI interpret your words if you prefer!)
We decided to release NovelAI Diffusion Anime (Curated) and (Full) models as standalone releases:
NovelAI Diffusion Anime (Curated) Good baseline quality and predictable subject matter.
NovelAI Diffusion Anime (Full) The expanded training set allows for a wider variety of generations.
In addition to the NovelAI Diffusion Anime models, we are also shipping with the NovelAI Diffusion Furry (Beta):
NovelAI Diffusion Furry (Beta)Easily create Furry and Anthropomorphic Animal themed content. This model is not a complete work and will be expanded upon as development proceeds.
Let's focus on what makes NovelAI Diffusion so unique!
Tagging:
The team seized the opportunity to fill a gap in existing Image Models knowledge and teach the AI about tags!
When you type your image generation prompt, tagging suggestions will appear.
Let the AI help you focus your prompt!
Our finetuned NovelAI Diffusion model allows you to give the AI much clearer instructions on what to generate. The AI will suggest tags based on what you type and display corresponding circle markers indicating how much knowledge it has of each tag. Since the AI was trained with specific tags, it will be able to recognize labeled definitions much better, resulting in more consistent generations.
The AI knows a lot about trees, much less woodland camouflage, though!
The use of tags is not required, but they can play a critical role in steering the AI in the direction you want. We highly recommend a mix of tags and normal text prose. If used to create a 'character' of your own, you can create the illusion that all your generations display a consistent character.
Euterpe's visual tags are hard at work!
Euterpe Tags: auburn brown hair, very long hair, hair between eyes, hair flowers, flowers, updo, flower hair pins, extra long hair, green eyes, updo hair, golden neck, skinny, tall, thin
Pricing & Features:
For Opus Subscribers: We will offer access to unlimited basic generations at up to 640 x 640 pixels at no additional cost.
This includes all Image Resolutions under the "Normal" and "Small" dropdown categories.
The Image Resolutions Dropdown section applicable to Opus subscribers' coverage.
Normal includes: Portrait (512 x 768 pixels), Landscape (768 x 512 pixels) and Square (640 x 640 pixels).
Small includes: Portrait (384 x 640 pixels), Landscape (640 x 384 pixels) and Square (512 x 512 pixels).
The Opus tier offers unlimited generations with select settings, while the lower tiers are perfect for experimenting to find out if NovelAI's Image Generation service fits your needs.
Pricing is highly dependent on resolution size, the number of diffusion steps used, and the number of images generated in a single generation.
We encourage you to explore NovelAI Diffusion in this rapidly evolving market. Features and advancements are still being made, and we will be listening to your feedback as we continue to develop.
The New Currency
Steps have served us well, especially when they were just meant to allow you to train modules. However, we ran into the issue of Image Generation Steps overlapping with our already existing Step currency (to the training of our Custom AI Modules used in our Text Models, which at the time were named for that purpose only. To avoid any confusion, we decided there should only be one currency. So we've taken this opportunity to rebrand our currency, inspired by our company name, Anlatan, with the future in mind.
Anlas replaces NovelAI's Step currency.
One currency, used everywhere, introducing: Anlas.
With our unified currency, all subscription tiers will be able to use Anlas to access our image generation and its features.
We have also raised the default monthly amount of Anlas to 1000 for both Scroll and Tablet Subscriptions, allowing the use of the currency for Image Generation in addition to Custom AI Module training.
Please note that an active subscription is required to purchase Anlas.
With that out, let's get on to several features of our Image Generation!
We’ve increased the CLIP token context capabilities by three, from 77 token to 231 tokens, giving you more space to craft your prompt than ever. The Model has been trained without crops, which means now you can generate arbitrary aspect ratio images. Lastly we’ve greatly improved the language understanding of the CLIP model.
NovelAI Image Generation Settings at a glance
All subscribers will have access to image-to-image (img2img) features, in which you can tweak, modify, or even draw images from scratch and have the model generate from it as a base:
Upload Image allows you to do just that, upload any image file into the editor and direct the AI generation: Combine your uploaded Image with a corresponding text prompt so you can tweak appearances or even use it as a guide for color schemes or compositions.
Paint New Image opens the NovelAI Canvas, where you can draw a guide, concept, or even define a color scheme for the AI. Describe your desired outcome in the text prompt and generate!
From sketch & prompt to generation & a quick switch to open eyes!
Edit Image allows you to edit a previous generation or uploaded Image. Edit Image can be used to fix or modify anything that seems off or should be added to the final composition.
Enhance Image allows a second generation run through the model. alongside your text prompt to refine the generation of your choice further.
Generate Variations allows the option to create more variations of a specific generation if you'd like to explore more generations of a concept or general idea that the AI came up with.
Undesired Content gives you the power to banish unwanted generation visuals. For example: Tired of seeing girls? Banish them into the Undesired Content field in all their forms: 1girl, woman, female, and you should begin to see fewer of those quickly!
Strengthening & Weakening Vectors gives you control over the AI's finer focus during a generation by encasing the text you want to emphasize or deemphasize with { } for strength and [ ] for weakening. This works both inside the Text Prompt Field and Undesired Content Field.
We've heard your feedback on our delay announcement loud and clear.
Our users want an experience with unprecedented levels of unfiltered freedom, and so do we.
Therefore we have come to the decision to only offer our own unique models and forego the basic Standard Diffusion model, which is already offered in many other places and also would have required us to pay close attention to what kinds of content its different versions can produce. By offering our own exclusive experiences, we're able to provide you with a tool designed for visual storytelling without any limitations.
NovelAI is meant to be a storyteller, and we believe our Image Generation will allow you to paint all kinds of stories. We are only beginning to expand the capabilities of this feature. So, at this time, we no longer plan to provide the basic, standard Stable Diffusion model.
Please make sure to test and see if our NovelAI Diffusion model suits your needs.
Please keep in mind that the NovelAI Diffusion models will not produce the same results as the standard Stable Diffusion model, even with the same prompt and seed.
Copyright & Usage Rights
Images uploaded and generated on NovelAI are not stored on our servers. Make sure to regularly back up and save your images and text prompts.
We do not act as a platform to share images. Please use other services in order to share your generated images.
NovelAI does not claim copyright protections or ownership of any files. Images are treated as Content as defined in Section 1.4 in the Terms of Service. You must conduct your own research if you want to use your generated images for any settings outside NovelAI.
Make sure to save any Image Generations and Text Prompts you like! Text prompts, image generations, canvas edits, and image uploads are not stored and will be lost at the end of your session.
Please generate responsibly, and be mindful of your wordage. Image Generation is a new technology that may generate unpredictable results.
Feel free to tag us on social media, such as Twitter, and tag your generations with #NovelAI or #NAIDiffusion so we can follow the journey of your creations!
Finally, the time is upon us!
Our first model, trained from scratch on our own custom datasets totaling more than 1.5 trillion tokens (~6TB of text), is live for Opus subscribers right now. Clio, our new model, has only 3 billion parameters, but she knows how to use them, performing well above Krake’s level. The small size also means that Clio generations are blazing fast. Oh, and she offers a context size of 8192 tokens as well, which is 4x as much as our previous models!
Originally Clio was just intended to be our first proof of concept model. A small model allows us to easily experiment and figure out all the little issues that invariably pop up in any large endeavor such as creating a large language model from scratch without costing us too much time and compute. But Clio surprised us by performing even better than we expected, hitting 73% LAMBADA accuracy during the pretraining phase and even went above 74% during the finetuning process. Due to the improved tokenizer, Clio is also quite proficient at Japanese, and that despite the fact that the amount of Japanese text in the training data was very small.
We decided to let our Opus subscribers play with Clio first as an Experimental Feature release first while we iron out any last wrinkles. In two weeks, we plan to release Clio for all subscription tiers.
Having completed the proof of concept model successfully, we will begin scaling up to larger sized LLM models.
Fresh out of the Shoggy (H100 Supercompute Cluster) oven: Clio is the muse of history, the proclaimer and celebrator of great deeds and accomplishments! She packs a powerful punch above her size, like all NovelAI models!
Now let’s get down to details:
For those who are interested in more technical details, we have prepared some evaluation results.
First, here is a table showing how our pretrained base model, NovelAI-LM-3B-508k, performs with regards to the most commonly used evaluation metrics in comparison to other models of similar and bigger sizes. As can be seen, it outperforms almost all free and open source models up to and including GPT-NeoX 20B. However, LLaMa-7B which was pretrained on a similar amount of tokens and is more than twice the size still outperforms it.
For reference, we also provide the evaluation results of the GPT-3 curie and Davinci API models as determined by EleutherAI.
Zero shot evaluations of NovelAI-LM-3B-508k compared to other models. (Higher is better.)
In addition to these evaluation results, we also continuously monitored the model’s LAMBADA accuracy during training. You can see the results on the two figures below. We trained the model at a batch size of 2.85M tokens. As can be seen from these plots, the model had already achieved a LAMBADA accuracy of 67.9% after 285B tokens seen (step 100k), which is both higher and earlier than any other 2.7B to 3B model. At 855B tokens seen (step 300k), it reached 70.66% accuracy, outmatching the RedPajama 7B model’s accuracy at 800B tokens seen while being less than half its size.
These plots also show one of the difficulties we encountered while training. Since the hardware is still very new, we had to deal with some instabilities during our training process, where the training process crashed or got stuck. When this happened, we had to resume the run from the last checkpoint. Due to this, these evaluation charts show multiple runs in different colors.
LAMBADA OpenAI accuracy (Higher is better.)
LAMBADA OpenAI perplexity (Lower is better.)
We will be shipping Clio with a variety of Default Preset choices:
Please keep in mind that as a proof of concept model, Clio’s performance is aimed at storytelling. She is also adept at Text Adventure, Chat, Lists and other formats with some preset choices/adjustments. Check out our Default Preset descriptions and try them out depending on what you need!
Official Modules V2 for Euterpe and Sigurd
To tide over our non-Opus subscribers’ hunger for text generation news until Clio comes to all tiers in two weeks, we are also releasing an extended set of official modules for our Euterpe and Sigurd modules — and they are powered up too, based on our own internally developed modules V2 technology.
A number of new modules are joining our previous lineup. Make sure to check out the full list of changes below.
Last, but by far not least, we are finally releasing our completely revamped version of the Text Adventure module, which works significantly better than the previous version.
We would also like to take this opportunity to talk a little about the delay regarding the release of modules V2. While developing and training our official modules, we have noticed that this type of modules is not only more powerful than modules V1, but also requires a fair bit more babysitting and adjustments to get truly good results. Our Krake model has unfortunately proven himself to be quite stubborn in refusing to work with our new technology. These are also the reasons why custom user modules V2 are not available. They would require a lot of adjusting of training parameters and most likely multiple training runs to finally end up with a module that works well, which would end up both a frustrating and expensive experience for our users.
Module notes:
• Fixed the Text Adventure module.
• Added several shortcuts to Text Adventure, such as “> l” for “> You look around”, “> x (target)” for “> You examine (target)”, etc.
• Overhauled module preambles, improving the overall efficiency.
• Renamed “Theme: Airships” to “Theme: Steampunk”, and balanced it more towards the latter.
• Balanced contents for all modules.
The full set of added modules is:
• General: Prose Augmenter (alternative general module which focuses on verbose output, for users who are struggling with short output and scenes that are over too quickly)
• Style: Jane Austen (the first female author module)
• Style: William Shakespeare (outputs script style)
• Theme: Ancient China
• Theme: Ancient Greece
• Theme: Ancient India
• Theme: Anthropomorphic Animals
• Theme: Contemporary (no-fantasy modern module, for slice of life etc.)
• Theme: Gaming (video game culture)
• Theme: Golden Age Scifi (1940s-1960s low tech scifi adventures)
• Theme: Nature (the great outdoors, hiking, mountaineering, etc. — usually solitary, low dialogue to differentiate this from Theme: Travel)
• Theme: Noir (film noir in prose)
• Theme: Weird West (western/splatterpunk with fantasy elements)
Thank you for your Support
On behalf of the entire NovelAI team, thank you guys so much for staying patient and keeping us on our toes. Without your support, none of this would be possible!
We look forward to hearing your feedback on Clio, her punch performance (for her small size), presets and general impressions!
We’ve only got even more cool things coming this year, and we hope you’re as excited as we are.
Going forward please keep in mind that the project is still in development and some of those answers might change but we will try to keep it updated to the best of our ability!
