Lucene.NET 4.8 is in beta - and we need your help!

English Posts, .NET, Lucene, Lucene.NET, Open-source

Comments

4 min read

I am happy to announce that after a long development cycle, Lucene.NET v4.8 is finally ready to be used by your projects.

As you are probably aware, the last release of Lucene.NET (3.0.3) was about 4 years ago, in October 2012. That's a long time for anything - and especially for a project like Lucene that's moving rapidly fast. Just to give you an idea, Lucene is what powering Elasticsearch - which was hardly a thing in October 2012...

The last couple of years we have been working on porting more recent bits of Lucene to the .NET ecosystem, and recently also getting ready for making it .NET Core compatible. And as of today you can start using Lucene.NET 4.8 from your applications.

Lucene.NET 4.8 is available on our myget feed, along with the accompanying sub-projects (see below):

https://www.myget.org/feed/lucene-net/package/nuget/Lucene.Net.

Some more details below, including a call for help.

Stability

The new release, even though still considered beta, is believed to be much more stable and production ready than the widely-used 3.0.3.

The main reason behind this claim is testability - we have a much better and more rigid test harness, where 3.0.3 hardly had any. Index corruption and multithreading bugs are well known issues with 3.0.3, which I personally heard from many users over the years - StackOverflow, Microsoft, RavenDB and others.

Version 4.8 is highly tested, with over 2000 passing tests, many of them randomized, and some of them run under extreme conditions. While few of those still occasionally fail - they test exreme scenarios and can at least highlight the issue and allow us to fix them before getting to an RC.

Bottom line, I am confident 4.8 is stable and given enough users confirm the same and help us fix the last remaining bits - we can proceed to an RC soon. Help us get there!

Why so long?

Because we are doing this in our free time. And because commercial users unfortunately won't contribute back or sponsor the effort in anyway. I wrote about this not long ago, but unfortunately it doesn't seem like it's going to change soon.

This release is made possible mainly thanks to the help of several core contributors, and a lot of coding time contributed by some Microsoft employees - with the blessing of the company. So thanks for that!

We will appreciate any help with actual development, or development sponsorship. If you can't sponsor or participate in development, please consider upgrading to 4.8 and giving us a shout in our mailing lists to let us know how it went.

Subprojects

In recent versions of Lucene (unlike the currently released 3.0.3), the software is made out of several sub-projects. The main one is Lucene.Net.Core, and the rest are things like Lucene.Net.Queries, Lucene.Net.Analysis, etc.

Don't let the terminology confuse you - some of the sub-projects are as crucial to working with Lucene.NET as the core itself. And we need all subprojects to be ported, tested, and ready to be used.

While work around the Core is mostly done (except from some code cleanup, .NETification and a few failing tests), many sub-projects are still mid-porting (partially ported, not fully tested, or sometimes not ported at all). This is our current focus, and will appreciate help with that as well.

.NET Core support

Lucene.NET 4.8 is almost ready to run on .NET Core - with most of the code made compatible and uncompatible dependencies removed. We are now working on the last bits (making the tests compatible too, and sub-projects compatibility) but generally speaking - we are already there.

Getting started

Packages are released for every build to our MyGet stream: https://www.myget.org/feed/lucene-net/package/nuget/Lucene.Net. See this guide for getting started with a custom MyGet feed.

To install Lucene.NET, you can simply install the Core package to your project - and then add any Lucene.NET sub-projects via additional Install-Package commands:

Install-Package Lucene.Net
Install-Package Lucene.Net.Analysis.Common
Install-Package Lucene.Net.Queries

All our source code is located in github, and development currently happens on the master branch: https://github.com/apache/lucenenet. Feel free to clone and send us PRs!

The mailing list is updated with progress, and things left to do. Feel free to reach out, get involved, and ask us any question: https://cwiki.apache.org/confluence/display/LUCENENET/Mailing+Lists.

I will be posting additional information, demos and sample code soon - watch this space..


Comments

  • Henk Mahieu

    I just wanted to thank you and the other contributors for the great work on Lucene.NET!

  • Vijay Santhanam

    This is excellent! I've started getting into lucene again thanks to .net core.

  • Martin

    Very cool - thank you. Are there any release notes to see whats new between 3.0.3 and 4.8 ?

    thank you

    • Itamar Syn-Hershko

      Hi Martin, there's plenty of stuff that's changed - but mostly the API is going to look and feel the same. I will be posting about the new API and publish several demos very soon, to help the learning experience and transition smoother.

  • Jeremy

    This is great news! I'm already talking with my team to see if we can get 4.8 into our project so we can give your our results.

    You guys rock!

  • Alex

    Are you going to leave the dependency to Microsoft.Bcl.Immutable in? This package seems to deprecated.

    On another note, would you suggest to use the beta version in production rather than 3.0.3?

    Thanks.

    • Itamar Syn-Hershko

      Good point on the Bcl dependency - I pushed a commit that removes it now. Good catch, thanks!

      And yes, I consider this beta version to be more stable than 3.0.3 - so I would recommend trying it out in development and staging for a while and if nothing terrible pops up - let us know so we could officially RC it and recommend production usages.

  • Itamar Syn-Hershko

    Good point on the Bcl dependency - I pushed a commit that removes it now. Good catch, thanks!

    And yes, I consider this beta version to be more stable than 3.0.3 - so I would recommend trying it out in development and staging for a while and if nothing terrible pops up - let us know so we could officially RC it and recommend production usages.

  • Stan

    Thanks a lot, that's awesome. I needed facets badly and will definitely check it out. A few questions:

    1. What is the best way to get the bits? I don't see a NuGet feed for packages other than Lucene.Net. Is it better to build from source?

    2. Are there tips for migrating from 3.0.3?

    -Stan

    • Itamar Syn-Hershko

      The blog post contains a link to a MyGet feed - you can get this bits there. We will release to nuget when we are officially RC. Building from source is always an option too of course.

      As for migrating from 3.0.3 - I will be posting guides about using the 4.8 APIs and include some guidance on migration. Generally speaking, there is not much a difference as long as you stick to the high level APIs.

  • Christ

    Awesome work! I was considering dropping Lucene Net altogether due to a lack of development. While dedicated search servers like Sphinx or SOLR have worked well for me, being able to embed the search engine inside the application makes deployment really simple.

    Any plans on pushing this to the 'official' NuGet channel as well? NuGet does support pre releases and this may help to get the word out.

    • Itamar Syn-Hershko

      The MyGet feed serves us well now, once we get to RC we will do a prerelease to nuget - I'd rather we focus the little resources we have on getting stuff done :) As always though, we welcome any contribution..

  • kiwi

    Great news! Thanks to you and everyone working on this excellent project. Been on 3.0.3 for several years (and worked very well), but will now start the move to 4.8.

  • Alex

    Looking forward to trying it, but having trouble with the package manage in Visual Studio. I keep getting: "'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3." When I try to browse the MyGet link.

    • Itamar Syn-Hershko

      I'm afraid I don't see this - the link in this blog post works...

    • jacob

      Alex . Use one of the following url's: https://www.myget.org/F/lucene-net/api/v3/index.json or https://www.myget.org/F/lucene-net/

  • Nazmul Junaid

    Hey great work guys!

    I'd love to use it but we are using a number of things from Lucene.Net.Contrib Highlighter, Spatial, SpellChecker etc is there any plans for their release?

    • Itamar Syn-Hershko

      Yes, some of the sub-projects have already been ported - and the ones you mentioned and haven't will be ported soon. We will be happy to receive contributions for that as well.

  • spoodey

    Great work! I have some issues with the ICU4NET-dependency for the Analysis module. Is it just me, or is this something you are working on?

    • Itamar Syn-Hershko

      I suspected that would eventually happen. Let's move it to the projects mailing list...

  • vijay mani

    whats new in new release of lucene.net

  • Dat Huynh

    Lucene.net 4.8 beta running oke on winform but asp.net not run. ICU4NET-dependency had problem.

    • Itamar Syn-Hershko

      Bug reports and support questions should be posted to the project's mailing lists. Thanks.

  • Jeroen De Smedt

    Hey,

    I'm new to Lucene, but we want to use it in our new project. We weren't going to use Lucene.Net as it seemed the project was dead, but then I found this blogpost. So we're going to start straight with Lucene.Net 4.8 in the development of the project (unless you say it would be better to use 3.0.3 for now?)

    As I'm implementing, I see that there are a lot of classes not included in v4.8 that were in 3.0.3, eg QueryParser. Comparing the 3.0.3 version with current Lucene java version, it seemed that there isn't much difference though. So I wondered why it's not included in the 4.8 (yet) ?

    • Are there breaking changes?
    • Are you transferring the classes one by one?
    • Is this just because there's just too much work?

    Anyway, glad to see there are still development on the Lucene.Net project . Maybe, when I'm a little more familiar with the Lucene story, I can contribute myself.

    Thanks in advance! Kind regards, Jeroen

    • Itamar Syn-Hershko

      You can and should start with 4.8. QueryParser was just ported and in a pull request. The rest of your questions we will be happy to answer in our mailing list.

  • Alex

    Thanks I figured it out. What wasn't clear to me was you have to download the packages first and then install. I was trying to put the URL directly into nugget as a package source.

  • Nati

    Can we use lucene as a datastore ?

  • Juan J. Sáez

    Has Lucene.NET 4.8 the same API than Lucene 4.8?

    • Itamar Syn-Hershko

      Yes. Please direct all technical questions to the project's mailing list.

  • Yaroslav

    Hello! Thank you for huge work. You mentioned .net core support however there is no corresponding target framework in nuget package. Is there a way to touch it?

    • Itamar

      Yes, you can send us a pull request with a fix. Please reach out in the dev@ mailing list for discussion and furtjer instructions.

  • Stuart Wilson

    Hey there,

    What assembly/namespace is queryparser in? Can't find it anywhere. I've got Lucene.Net, Lucene.Net.Analysis.Common and Lucene.Net.Queries v4.8.0.685-beta installed?

    Many thanks

  • David Suarez

    Hi,

    I wanted to give this a go under .NET Core. I grabbed the latest code here https://github.com/apache/lucenenet (master), but didn't find any dotnetcore projects. I have created one and added just the lucene core. After fixing some 200 build errors, I'm still confronted with another 140.

    From your comments "Lucene.NET 4.8 is almost ready to run on .NET Core" above, I'm wondering if there is some code I'm missing? Seems that what is on master, while not using many dependencies, still relies heavely on big "non-portable" features, like serialization and others that could be fixed, but still require some amount of work.

    Can you please comment on the current dotnetcore port status and maybe give some pointers on the right direction if one would like to contribute to this effort?

    Thanks,

    David

    • Itamar Syn-Hershko

      Hi, the appropriate place for such discussions is the project's dev@ mailing list. Please hit us there and we will make sure to help.

Comments are now closed