MROONGA(1)

NAME
mroonga - Mroonga Documentation

For the release history: News <>

THE CHARACTERISTICS OF MROONGA

  What is Mroonga?

Mroonga is a MySQL storage engine based on Groonga, the full text search engine.

In MySQL 5.1 or later, Pluggable Storage Engine interface is introduced, and we can use custom storage engines easily. So we implement Mroonga, so that we can use Groonga through MySQL.

By using Mroonga, you can use Groonga with SQL.

  The successor of Tritonn

To support Japanese full text search, Tritonn was developed by embedding Senna, the predecessor of Groonga, in MySQL. Mroogna is its successor.

  Running as a MySQL plugin

Since Tritonn was the modified version of MySQL, we need to build it by ourselves or use binary files provided by Tritonn project, thus we cannot use the official binary files provided by MySQL.

On the other hand, Mroonga is an independent program (shared library) using Pluggable Storage Engine interface, and we can dynamically load it on MySQL's official binary. So we can use it more easily than Tritonn.

  Faster index update

Comparing to Senna, Groonga has much better throughput in adding or updating index.

Mroonga also has the benefit of this performance improvement.

  Faster search

In Tritonn, we use MyISAM storage engine, thus we have a exclusive table lock by updating data (and index), and it prevents the performance of search.

But in Mroonga, we no longer have this issue, and the performance of search is better especially in frequent data update cases.

  Geolocation search

Groonga supports not only the full text search, but also the fast geolocation search using index. And MySQL also has the syntax for geolocation search. With Mroonga, you can use Groonga's fast geolocation search by using MySQL's geolocation SQL syntax.

  Sharing the same Groonga storage

Mroonga stores the data by using Groonga's DB API. And its storage file's format is same as that of the file that is managed by Groonga itself only. Therefore you can share the same Groonga storage like below.

•   Store data through Mroonga (MySQL) and search from Groonga server.

•   Store data through Groonga server and search from Mroonga (MySQL).

    And Groonga's storage file can be shared with multi-processes and multi-threads, so that we can invoke several search queries to the same storage file simultaneously.

  Associate with other storage engines

Mroonga has two running modes.

One is "storage mode", that is the default mode, and we use Groonga for both storing data and searching. With this mode, you can have full benefits of Groonga described above, like fast data update, lock-free full text search and geolocation search. But it does not support transactions.

Another one is "wrapper mode", that adds full text search function on other storage engines like MyISAM or InnoDB. With this mode, you can use Groonga's fast full text search with having the benefits of the storage engine, ex. transaction in InnoDB. But you cannot have benefits from Groonga's read-lock free characteristic. And you might have the performance bottle neck in the storage engine in updating data.

  Supported platforms

Mroonga supports many of the major platforms.

Refer to Install <> for supported platforms.

INSTALL

This section describes how to install Mroonga on each environment. There are packages for major platforms. It's recommended that you use package instead of building Mroonga by yourself. But don't worry. There is a document about building Mroonga from source.

We distribute both 32-bit and 64-bit packages but we strongly recommend a 64-bit package for server. You should use a 32-bit package just only for tests or development. You will encounter an out of memory error with a 32-bit package even if you just process medium size data.

  Windows

This section describes how to install Mroonga on Windows. You can install Mroogna by extracting a zip package.

Mroonga binary for Windows is provided with MariaDB binary because some changes <https://github.com/mroonga/mroonga/tree/main/packages/source/patches> are needed for building Mroonga for Windows.

  Install

  Download and extract the zip file

Download zip file and extract it. You need to choose a zip for your environment:

•   MariaDB with Mroonga-16.01 <https://github.com/mroonga/mroonga/releases/tag/v16.01>

  Run mariadb-install-db.exe

Run bin\mariadb-install-db.exe to initialize the database. mariadb-install-db.exe is included in the zip file.

    > bin\mariadb-install-db.exe --datadir=C:\EXAMPLE\data --service=MariaDB --password=PASSWORD

•   --datadir=C:\EXAMPLE\data

•   Data directory of the new database

•   --service=MariaDB

•   Name of the Windows service

•   Specify if you want to register MariaDB as a Windows service

•   --password=PASSWORD

•   Password of the root user

  Start MariaDB server command

Just start MariaDB server by following command.

    > bin\mariadbd.exe --defaults-file=.\MY-PREFERRED-INI.ini --console

If MariaDB is registered as a Windows service, it will be started by the Windows service.

  Install Mroonga to MariaDB

Next connect to MariaDB by following command.

    > bin\mariadb.exe
    MariaDB [(none)]>

After connecting, execute share\mroonga\install.sql to install Mroonga. share\mroonga\install.sql is included in the zip file and should be specified with an appropriate path.

    MariaDB [(none)]> SOURCE C:PATHTO\share\mroonga\install.sql;
    Query OK, 0 rows affected (0.064 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.002 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.001 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.001 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.000 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.001 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.000 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.000 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.001 sec)
    Query OK, 0 rows affected, 1 warning (0.000 sec)
    Query OK, 0 rows affected (0.000 sec)
    MariaDB [(none)]> SHOW ENGINES;
    +--------------------+---------+------------------------------------------------------------+--------------+------+------------+
    | Engine             | Support | Comment                                                    | Transactions | XA   | Savepoints |
    +--------------------+---------+------------------------------------------------------------+--------------+------+------------+
    | CSV                | YES     | CSV storage engine                                         | NO           | NO   | NO         |
    | PERFORMANCE_SCHEMA | YES     | Performance Schema                                         | NO           | NO   | NO         |
    | MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables  | NO           | NO   | NO         |
    | MyISAM             | YES     | MyISAM storage engine                                      | NO           | NO   | NO         |
    | MRG_MyISAM         | YES     | Collection of identical MyISAM tables                      | NO           | NO   | NO         |
    | InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES          | YES  | YES        |
    | Mroonga            | YES     | CJK-ready fulltext search, column store                    | NO           | NO   | NO         |
    | Aria               | YES     | Crash-safe tables with MyISAM heritage                     | NO           | NO   | NO         |
    +--------------------+---------+------------------------------------------------------------+--------------+------+------------+
    8 rows in set (0.00 sec)

  Build from source with MariaDB

You need to use Mroonga bundled MariaDB source provided by the Mroonga project.

You can find it in <https://packages.groonga.org/source/mroonga/>. Mroonga bundled MariaDB source has mariadb-${MARIADB_VERSION}-with-mroonga-${MROONGA_VERSION}.zip file name.

You can build the source code with the standard MariaDB build process <https://mariadb.com/kb/en/mariadb/documentation/getting-started/compiling-mariadb-from-source/Building_MariaDB_on_Windows/>.

You need to register Mroonga after building MariaDB. Use SQL at ${MARIADB_BUILD_DIR}\storage\mroonga\data\install.sql to register Mroonga.

  macOS

This section describes how to install Mroonga on macOS. You can install Mroonga by Homebrew <https://brew.sh/>.

  Homebrew

See mroonga/homebrew-mroonga on GitHub <https://github.com/mroonga/homebrew-mroonga> for details.

  Debian GNU/Linux

This section describes how to install Mroonga related deb packages on Debian GNU/Linux. You can install them by apt.

  bookworm (MariaDB)

Install:

    $ sudo apt update
    $ sudo apt install -y -V wget
    $ wget https://packages.apache.org/artifactory/arrow/debian/apache-arrow-apt-source-latest-bookworm.deb
    $ sudo apt install -y -V ./apache-arrow-apt-source-latest-bookworm.deb
    $ rm -rf apache-arrow-apt-source-latest-bookworm.deb
    $ wget https://packages.groonga.org/debian/groonga-apt-source-latest-bookworm.deb
    $ sudo apt install -y -V ./groonga-apt-source-latest-bookworm.deb
    $ rm -rf groonga-apt-source-latest-bookworm.deb
    $ sudo apt update
    $ sudo apt install -y -V mariadb-10.11-mroonga

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo apt install -y -V groonga-tokenizer-mecab

  bookworm (with the Oracle MySQL 8.0 package)

Install:

    $ sudo apt update
    $ sudo apt install -y -V wget
    $ wget https://packages.groonga.org/debian/groonga-apt-source-latest-bookworm.deb
    $ sudo apt install -y -V ./groonga-apt-source-latest-bookworm.deb
    $ rm -rf groonga-apt-source-latest-bookworm.deb
    $ wget https://repo.mysql.com/mysql-apt-config.deb
    $ sudo env DEBIAN_FRONTEND=noninteractive MYSQL_SERVER_VERSION=mysql-8.0 apt install -y ./mysql-apt-config.deb
    $ rm -rf mysql-apt-config.deb
    $ sudo apt update
    $ sudo apt install -y -V mysql-community-8.0-mroonga

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo apt install -y -V groonga-tokenizer-mecab

  trixie (MariaDB)

Install:

    $ sudo apt update
    $ sudo apt install -y -V wget
    $ wget https://packages.apache.org/artifactory/arrow/debian/apache-arrow-apt-source-latest-trixie.deb
    $ sudo apt install -y -V ./apache-arrow-apt-source-latest-trixie.deb
    $ rm -rf apache-arrow-apt-source-latest-trixie.deb
    $ wget https://packages.groonga.org/debian/groonga-apt-source-latest-trixie.deb
    $ sudo apt install -y -V ./groonga-apt-source-latest-trixie.deb
    $ rm -rf groonga-apt-source-latest-trixie.deb
    $ sudo apt update
    $ sudo apt install -y -V mariadb-11.8-mroonga

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo apt install -y -V groonga-tokenizer-mecab

  trixie (with the Oracle MySQL 8.4 package)

Install:

    $ sudo apt update
    $ sudo apt install -y -V wget
    $ wget https://packages.groonga.org/debian/groonga-apt-source-latest-trixie.deb
    $ sudo apt install -y -V ./groonga-apt-source-latest-trixie.deb
    $ rm -rf groonga-apt-source-latest-trixie.deb
    $ wget https://repo.mysql.com/mysql-apt-config.deb
    $ sudo env DEBIAN_FRONTEND=noninteractive MYSQL_SERVER_VERSION=mysql-8.4-lts apt install -y ./mysql-apt-config.deb
    $ rm -rf mysql-apt-config.deb
    $ sudo apt update
    $ sudo apt install -y -V mysql-community-8.4-mroonga

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo apt install -y -V groonga-tokenizer-mecab

  Ubuntu

This section describes how to install Mroonga related deb packages on Ubuntu. You can install them by apt.

  PPA (Personal Package Archive)

The Mroonga APT repository for Ubuntu uses PPA (Personal Package Archive) on Launchpad. You can install Mroonga by APT from the PPA.

Here are supported Ubuntu versions:

•   22.04 Jammy Jellyfish

•   24.04 Noble Numbat

    Here are Ubuntu versions that supports MySQL:

•   22.04 Jammy Jellyfish

•   24.04 Noble Numbat

    Here are Ubuntu versions that supports MariaDB:

•   22.04 Jammy Jellyfish

•   24.04 Noble Numbat

    Enable the universe repository and the security update repository to install Mroonga:

        % sudo apt-get install -y -V software-properties-common lsb-release
        % sudo add-apt-repository -y universe
        % sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu $(lsb_release --short --codename)-security main restricted"

    Add the ppa:groonga/ppa PPA to your system:

        % sudo add-apt-repository -y ppa:groonga/ppa
        % sudo apt-get update

    Install Mroonga for MySQL:

        % sudo apt-get install -y -V mysql-server-mroonga

    Install Mroonga for MariaDB:

        % sudo apt-get install -y -V mariadb-server-mroonga

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        % sudo apt-get install -y -V groonga-tokenizer-mecab

  AlmaLinux

This section describes how to install Mroonga related RPM packages on AlmaLinux. You can install them by dnf.

  AlmaLinux 8 (with the Oracle MySQL 8.0 package)

You can use Oracle's MySQL packages version 8.0 on AlmaLinux 8 since Mroonga 11.10 release.

Note: There are already known issues about MySQL 8.0.

•   Wrapper mode <> Wrapper mode is not supported yet

•   Storage mode <> Storage mode does not support the following feature.

•   The feature of relevant to the optimization.

    Install:

        $ sudo dnf -y module disable mysql
        $ sudo dnf install -y  https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
        $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
        $ sudo dnf install -y https://repo.mysql.com/mysql80-community-release-el8.rpm
        $ sudo dnf install --disablerepo=AppStream -y --enablerepo=epel,powertools mysql-community-8.0-mroonga
        ($ sudo systemctl start mysqld)
        ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
        ($ sudo mysqladmin -u root --password="${tmp_password}" password)

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        $ sudo dnf -y module enable mysql
        $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab
        $ sudo dnf -y module disable mysql

  AlmaLinux 8 (with the Oracle MySQL 8.4 package)

You can use Oracle's MySQL packages version 8.4 on AlmaLinux 8 since Mroonga 14.04 release.

Note: There are already known issues about MySQL 8.4.

•   Wrapper mode <> Wrapper mode is not supported

•   Storage mode <> Storage mode does not support a feature relevant to an optimization.

    Install:

        $ sudo dnf -y module disable mysql
        $ sudo dnf install -y  https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
        $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
        $ sudo dnf install -y https://repo.mysql.com/mysql84-community-release-el8.rpm
        $ sudo dnf install -y --enablerepo=epel,powertools mysql-community-8.4-mroonga
        ($ sudo systemctl start mysqld)
        ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
        ($ sudo mysqladmin -u root --password="${tmp_password}" password)

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        $ sudo dnf -y module enable mysql
        $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab
        $ sudo dnf -y module disable mysql

  AlmaLinux 8 (with Percona Server 8.0 package)

You can use Percona Server packages version 8.0 on AlmaLinux 8 since Mroonga 11.10 release.

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
    $ sudo dnf install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm
    $ sudo percona-release setup ps80
    $ sudo dnf install -y --enablerepo=epel,powertools percona-server-8.0-mroonga
    ($ sudo systemctl start mysqld)
    ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
    ($ sudo mysqladmin -u root --password="${tmp_password}" password)

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 8 (with MariaDB 10.6 package)

You can use MariaDB's MariaDB packages version 10.6 on AlmaLinux 8 since Mroonga 11.10 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/10.6/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
    $ sudo dnf module -y disable mariadb
    $ sudo dnf module -y disable mysql
    $ sudo dnf install -y --enablerepo=powertools mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y --enablerepo=powertools mariadb-10.6-mroonga
    ($ sudo mysqladmin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf module -y enable mysql
    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab
    $ sudo dnf module -y disable mysql

  AlmaLinux 8 (with MariaDB 10.11 package)

You can use MariaDB's MariaDB packages version 10.11 on AlmaLinux 8 since Mroonga 13.01 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/10.11/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
    $ sudo dnf module -y disable mariadb
    $ sudo dnf module -y disable mysql
    $ sudo dnf install -y --enablerepo=powertools mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y --enablerepo=powertools mariadb-10.11-mroonga
    ($ sudo mysqladmin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf module -y enable mysql
    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab
    $ sudo dnf module -y disable mysql

  AlmaLinux 8 (with MariaDB 11.4 package)

You can use MariaDB's MariaDB packages version 11.4 on AlmaLinux 8 since Mroonga 14.07 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/11.4/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
    $ sudo dnf module -y disable mariadb
    $ sudo dnf module -y disable mysql
    $ sudo dnf install -y --enablerepo=powertools mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y --enablerepo=powertools mariadb-11.4-mroonga
    ($ sudo mariadb-admin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf module -y enable mysql
    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab
    $ sudo dnf module -y disable mysql

  AlmaLinux 8 (with MariaDB 11.8 package)

You can use MariaDB's MariaDB packages version 11.8 on AlmaLinux 8 since Mroonga 15.21 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/11.8/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/8/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
    $ sudo dnf module -y disable mariadb
    $ sudo dnf module -y disable mysql
    $ sudo dnf install -y --enablerepo=powertools mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y --enablerepo=powertools mariadb-11.8-mroonga
    ($ sudo mariadb-admin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf module -y enable mysql
    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab
    $ sudo dnf module -y disable mysql

  AlmaLinux 9 (with the Oracle MySQL 8.0 package)

You can use Oracle's MySQL packages version 8.0 on AlmaLinux 9 since Mroonga 12.12 release.

Note: There are already known issues about MySQL 8.0.

•   Wrapper mode <> Wrapper mode is not supported yet

•   Storage mode <> Storage mode does not support the following feature.

•   The feature of relevant to the optimization.

    Install:

        $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
        $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
        $ sudo dnf install -y https://repo.mysql.com/mysql80-community-release-el9.rpm
        $ sudo dnf install --disablerepo=AppStream -y --enablerepo=epel,crb mysql-community-8.0-mroonga
        ($ sudo systemctl start mysqld)
        ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
        ($ sudo mysqladmin -u root --password="${tmp_password}" password)

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 9 (with the Oracle MySQL 8.4 package)

You can use Oracle's MySQL packages version 8.4 on AlmaLinux 9 since Mroonga 14.04 release.

Note: There are already known issues about MySQL 8.4.

•   Wrapper mode <> Wrapper mode is not supported

•   Storage mode <> Storage mode does not support a feature relevant to an optimization.

    Install:

        $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
        $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
        $ sudo dnf install -y https://repo.mysql.com/mysql84-community-release-el9.rpm
        $ sudo dnf install -y --enablerepo=epel,crb mysql-community-8.4-mroonga
        ($ sudo systemctl start mysqld)
        ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
        ($ sudo mysqladmin -u root --password="${tmp_password}" password)

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 9 (with Percona Server 8.0 package)

You can use Percona Server packages version 8.0 on AlmaLinux 9 since Mroonga 12.12 release.

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
    $ sudo dnf install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm
    $ sudo percona-release setup ps80
    $ sudo dnf install -y --enablerepo=epel percona-server-8.0-mroonga
    ($ sudo systemctl start mysqld)
    ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
    ($ sudo mysqladmin -u root --password="${tmp_password}" password)

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 9 (with Percona Server 8.4 package)

You can use Percona Server packages version 8.4 on AlmaLinux 9 since Mroonga 15.04 release.

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
    $ sudo dnf install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm
    $ sudo percona-release enable-only ps-84-lts release
    $ sudo dnf install -y --enablerepo=epel percona-server-8.4-mroonga
    ($ sudo systemctl start mysqld)
    ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
    ($ sudo mysqladmin -u root --password="${tmp_password}" password)

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 9 (with MariaDB 10.6 package)

You can use MariaDB's MariaDB packages version 10.6 on AlmaLinux 9 since Mroonga 12.12 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/10.6/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
    $ sudo dnf install -y mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y mariadb-10.6-mroonga
    ($ sudo mysqladmin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 9 (with MariaDB 10.11 package)

You can use MariaDB's MariaDB packages version 10.11 on AlmaLinux 9 since Mroonga 13.01 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/10.11/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
    $ sudo dnf install -y mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y mariadb-10.11-mroonga
    ($ sudo mysqladmin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 9 (with MariaDB 11.4 package)

You can use MariaDB's MariaDB packages version 11.4 on AlmaLinux 9 since Mroonga 14.07 release.

Create /etc/yum.repos.d/MariaDB.repo with the following content:

    [mariadb]
    name = MariaDB
    baseurl = https://rpm.mariadb.org/11.4/rhel/$releasever/$basearch
    gpgkey = https://rpm.mariadb.org/RPM-GPG-KEY-MariaDB
    gpgcheck = 1

Install:

    $ sudo dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/9/apache-arrow-release-latest.rpm
    $ sudo dnf install -y https://packages.groonga.org/almalinux/9/groonga-release-latest.noarch.rpm
    $ sudo dnf install -y mariadb-server
    $ sudo systemctl start mariadb
    $ sudo dnf install -y mariadb-11.4-mroonga
    ($ sudo mariadb-admin -u root password 'new-password')

If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

Install groonga-tokenizer-mecab package:

    $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  AlmaLinux 10 (with the Oracle MySQL 8.4 package)

You can use Oracle's MySQL packages version 8.4 on AlmaLinux 10 since Mroonga 15.18 release.

Note:

•   Wrapper mode <> Wrapper mode is not supported

•   Storage mode <> Storage mode does not support a feature relevant to an optimization.

    Install:

        $ sudo dnf install -y  https://packages.apache.org/artifactory/arrow/almalinux/10/apache-arrow-release-latest.rpm
        $ sudo dnf install -y https://packages.groonga.org/almalinux/10/groonga-release-latest.noarch.rpm
        $ sudo dnf install -y https://repo.mysql.com/mysql84-community-release-el10.rpm
        $ sudo dnf install --disablerepo=AppStream -y --enablerepo=epel mysql-community-8.4-mroonga
        ($ sudo systemctl start mysqld)
        ($ tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
        ($ sudo mysqladmin -u root --password="${tmp_password}" password)

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        $ sudo dnf install -y --enablerepo=epel groonga-tokenizer-mecab

  Oracle Linux

This section describes how to install Mroonga related RPM packages on AlmaLinux. You can install them by dnf.

  Oracle Linux 8 (with the Oracle MySQL 8.0 package)

You can use Oracle's MySQL packages version 8.0 on Oracle Linux 8 since Mroonga 12.08 release.

Note: There are already known issues about MySQL 8.0.

•   Wrapper mode <> Wrapper mode is not supported yet

•   Storage mode <> Storage mode does not support the following feature.

•   The feature of relevant to the optimization.

    Install:

        % sudo dnf -y module disable mysql
        % sudo dnf install -y https://packages.groonga.org/almalinux/8/groonga-release-latest.noarch.rpm
        % sudo dnf install -y https://repo.mysql.com/mysql80-community-release-el8.rpm
        % sudo dnf install --disablerepo=AppStream -y --enablerepo=ol8_codeready_builder mysql-community-8.0-mroonga
        (% sudo systemctl start mysqld)
        (% tmp_password=$(sudo grep 'A temporary password' /var/log/mysqld.log | sed -e 's/^.*: //'))
        (% sudo mysqladmin -u root --password="${tmp_password}" password)

    If you want to use MeCab <https://taku910.github.io/mecab/> as a tokenizer, install groonga-tokenizer-mecab package.

    Install groonga-tokenizer-mecab package:

        % sudo dnf install -y --enablerepo=ol8_codeready_builder groonga-tokenizer-mecab

  Fedora

  Fedora 21

MariaDB 10.0.x is adopted on Fedora 21. As Mroonga is now bundled with MariaDB since 10.0.15, you can use bundled version of Mroonga.

Note that MariaDB bundled Mroonga version is a bit old.

  Docker

Mroonga is also available via Docker Hub.

See Quick Start <https://hub.docker.com/r/groonga/mroonga/> guide to start.

  Others

This section describes how to install Mroonga from source code. If there is no package no your environment, you need to build Mroonga from source code.

  Dependencies

Mroonga needs some tools, libraries and MySQL for build. You can use MariaDB instead of MySQL.

  Tools

Here are required tools.

•   wget, curl or Web browser for downloading source archive

•   tar and gzip for extracting source archive

•   shell (many shells such as dash, bash and zsh will work)

•   C compiler and C++ compiler (gcc and g++ are supported but other compilers may work)

•   CMake <https://cmake.org/> as a cross-platform build system generator

•   Ninja <https://ninja-build.org/> as a small build system with a focus on speed

•   pkg-config <http://www.freedesktop.org/wiki/Software/pkg-config> for detecting libraries

    You must get them ready.

    Here are optional tools.

•   sudo for installing built Mroonga

  Libraries

Here are required libraries.

•   Groonga <https://groonga.org/>

•   If you use package, install development package such as libgroonga-dev for deb or groonga-devel for RPM

•   groonga-normalizer-mysql <https://github.com/groonga/groonga-normalizer-mysql>

    Here are optional libraries.

•   MeCab <https://taku910.github.io/mecab/>: Japanese morphological analysis system

    Note: If you want to use indexes of tokenizing of each morpheme for full text search, install MeCab before installing Groonga.

  MySQL

Mroonga needs not only installed MySQL but also MySQL source and build directory. You can't use MySQL package. It doesn't provide MySQL source and build directory. You need MySQL source and build directory!

If you use MariaDB instead of MySQL, you need MariaDB source.

Download the latest MySQL 8.4 source code, then build and install it.

See also Download MySQL Community Server <http://dev.mysql.com/downloads/mysql/>

Here we assume that you use mysql-8.4.1 and its source code is extracted in the following directory.

    ${HOME}/local/src/mysql-8.4.1

Then build in the following directory.

    ${HOME}/local/build/mysql-8.4.1

Here are command lines to build and install MySQL.

    $ cmake \
        -S ${HOME}/local/src/mysql-8.4.1 \
        -B ${HOME}/local/build/mysql-8.4.1 \
        -GNinja \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=${HOME}/local
    $ cmake --build ${HOME}/local/build/mysql-8.4.1
    $ cmake --install ${HOME}/local/build/mysql-8.4.1

You need to run MySQL before you install Mroonga. Because you need to run some SQL statements to register Mroonga.

  MariaDB

You can use MariaDB instead of MySQL.

Note that you need to remove storage/mroonga/ (Mroonga bundled in MariaDB) before you build MariaDB.

    $ mkdir -p ${HOME}/local/src
    $ cd ${HOME}/local/src
    $ wget https://downloads.mariadb.org/rest-api/mariadb/11.4.3/mariadb-11.4.3.tar.gz
    $ tar xf mariadb-11.4.3.tar.gz
    $ rm -rf mariadb-11.4.3/storage/mroonga
    $ cd -
    $ cmake \
        -S ${HOME}/local/src/mariadb-11.4.3 \
        -B ${HOME}/local/build/mariadb-11.4.3 \
        -GNinja \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=${HOME}/local
    $ cmake --build ${HOME}/local/build/mariadb-11.4.3
    $ cmake --install ${HOME}/local/build/mariadb-11.4.3

  Build from source

Mroonga uses CMake. So the following is the simplest build steps.

    $ cd ${HOME}/local/src
    $ wget https://packages.groonga.org/source/mroonga/mroonga-latest.tar.gz
    $ tar xvf mroonga-latest.tar.gz
    $ mroonga_base_name=$(find mroonga-* -maxdepth 0 -type d)
    $ cmake \
        -S ${HOME}/local/src/${mroonga_base_name} \
        -B ${HOME}/local/build/${mroonga_base_name} \
        -GNinja \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=${HOME}/local \
        -DMYSQL_BUILD_DIR=${HOME}/local/build/mysql-8.4.1 \
        -DMYSQL_CONFIG=${HOME}/local/bin/mysql_config \
        -DMYSQL_SOURCE_DIR=${HOME}/local/src/mysql-8.4.1
    $ cmake --build ${HOME}/local/build/${mroonga_base_name}
    $ cmake --install ${HOME}/local/build/${mroonga_base_name}
    $ ${HOME}/local/bin/mysql -u root < ${HOME}/local/share/mroonga/install.sql

You need to specify the following on cmake.

•   -DMYSQL_BUILD_DIR: The location of MySQL build directory

•   -DMYSQL_CONFIG: The path of mysql_config command

•   -DMYSQL_SOURCE_DIR: The location of MySQL source code

    You can confirm Mroonga is installed successfully by SHOW ENGINES SQL. If you can find Mroonga row, Mroonga is installed successfully.

        mysql> SHOW ENGINES;
        +------------+---------+------------------------------------------------------------+--------------+------+------------+
        | Engine     | Support | Comment                                                    | Transactions | XA   | Savepoints |
        +------------+---------+------------------------------------------------------------+--------------+------+------------+
        | Mroonga    | YES     | Fulltext search, column base                               | NO           | NO   | NO         |
        | MRG_MYISAM | YES     | Collection of identical MyISAM tables                      | NO           | NO   | NO         |
        | CSV        | YES     | CSV storage engine                                         | NO           | NO   | NO         |
        | MyISAM     | DEFAULT | Default engine as of MySQL 3.23 with great performance     | NO           | NO   | NO         |
        | InnoDB     | YES     | Supports transactions, row-level locking, and foreign keys | YES          | YES  | YES        |
        | MEMORY     | YES     | Hash based, stored in memory, useful for temporary tables  | NO           | NO   | NO         |
        +------------+---------+------------------------------------------------------------+--------------+------+------------+
        6 rows in set (0.00 sec)

    The following describes details about each step.

  cmake

First, you need to run cmake. Here are important cmake parameters.

  -DMYSQL_SOURCE_DIR=PATH

Specifies the location of MySQL source code.

This is required parameter.

  -DMYSQL_BUILD_DIR=PATH

Specifies the location where you build MySQL source code.

If you build MySQL in MySQL source code directory, you don't need to specify this parameter. If you build MySQL in other directory, you need to specify this parameter.

  -DMYSQL_CONFIG=PATH

Specifies the path of mysql_config command.

If mysql_config command can be found by PATH, you don't need to specify this parameter. For example, if mysql_config command exists at /usr/bin/mysql_config, you don't need to specify this parameter.

  -DMRN_DEFAULT_TOKENIZER=TOKENIZER

Specifies the default tokenizer for full text. You can custom it in my.cnf.

The default is TokenBigram.

Here is an example to use TokenMecab as the default tokenizer:

    $ cmake ... -DMRN_DEFAULT_TOKENIZER=TokenMecab

  -DCMAKE_BUILD_TYPE={Release,Debug,RelWithDebInfo}

Specifies the build type.

If you use this Mroonga as a production environment, you must use -DCAMKE_BUILD_TYPE=Release or -DCAMKE_BUILD_TYPE=RelWithDebInfo. They enable optimization.

If you use this Mroonga for development, you must use -DCAMKE_BUILD_TYPE=Debug. It disables optimization and enables debug symbols. They are useful for development.

  -DCMAKE_INSTALL_PREFIX=PREFIX

Specifies the install base directory. Mroonga related files are installed under ${PREFIX}/ directory except ha_mroonga.so. ha_mroonga.so is a MySQL plugin file. It is installed the plugin directory of MySQL.

The default is /usr/local. In this case, install.sql that is used for installing Mroonga is installed to /usr/local/share/mroonga/install.sql.

  cmake --build

If cmake is succeeded, you can build Mroonga by cmake --build.

If you get some errors by cmake --build, please report them to us: How to report a bug <>

  cmake --install

Now, you can install built Mroonga!

If you don't have write permission for ${PREFIX} and the plugin directory of MySQL, you need to use sudo:

    $ sudo cmake --install ${HOME}/local/build/${mroonga_base_name}

  mysql -u root < install.sql

You need to run some SQL statements to register Mroonga to MySQL such as INSTALL PLUGIN and CREATE FUNCTION. They are written in ${PREFIX}/share/mroonga/install.sql.

  Uninstall Mroonga

If you want to uninstall Mroonga, use the following command lines:

    $ ${HOME}/local/bin/mysql -u root < ${PREFIX}/share/mroonga/uninstall.sql
    $ xargs rm < ${HOME}/local/build/mroonga/install_manifest.txt

UPGRADE

There is a case that incompatible change is introduced at new release. It is announced by release announce if new release contains such a incompatible change.

Here is the list of recommended way of upgrading Mroonga from old release.

See following URL about upgrade sequence if you use previous version.

If you upgrade prior to 1.20, refer to Release 1.20 - 2012/01/29 <#release-1-20>

If you upgrade from 1.20, refer to Release 2.00 - 2012/02/29 <#release-2-00>

If you upgrade from 2.00 or 2.01, refer to Release 2.02 - 2012/04/29 <#release-2-02>

If you upgrade from 2.00 or later and using multiple column indexes on storage mode, refer to Release 2.03 - 2012/05/29 <#release-2-03>

If you upgrade from 2.04 or later and using SET column or ENUM that has the number of elements < 256 in Storage mode, refer to Release 2.05 - 2012/07/29 <#release-2-05>

If you upgrade from 2.05 or later and using multiple column indexes against VARCHAR or CHAR, refer to Release 2.08 - 2012/10/29 <#release-2-08>

If you upgrade from 2.08 or later and using TIMESTAMP column, please recreate database. If you upgrade from 2.08 or later and using CHAR(N) as primary key, please recreate index. Refer to Release 2.09 - 2012/11/29 <#release-2-09> for each case.

If you upgrade prior to 5.03 and satisfies following the conditions, refer to Release 5.04 - 2015/06/29 <#release-5-04> and upgrade schema.

•   Using custom tokenizer in index comment

•   Using mroonga_default_parser <#server-variable-mroonga-default-parser> as server variable

•   Using index_flags parameter for index column

•   Using type parameter for Groonga's column type in storage mode

TUTORIAL

If you don't install Mroonga yet, see Install <>. It describes how to install Mroonga.

Try the following tutorials after you install Mroonga. You will understand how to use Mroonga.

  Installation check

It is better that you check Mroonga installation before you use Mroonga. If Mroonga installation is failed, the SQLs in this tutorial will fail.

The way to start or stop MySQL server is just same as the normal MySQL.

After invoking the MySQL server, connect to it by mysql command. If you set password, you need to add '-p' option.

    % mysql -uroot test

By using SHOW ENGINES command, you can check if Mroonga is installed:

    SHOW ENGINES;
    -- +------------+---------+------------------------------------------------------------+--------------+------+------------+
    -- | Engine     | Support | Comment                                                    | Transactions | XA   | Savepoints |
    -- +------------+---------+------------------------------------------------------------+--------------+------+------------+
    -- | Mroonga    | YES     | Fulltext search, column base                               | NO           | NO   | NO         |
    -- | MRG_MYISAM | YES     | Collection of identical MyISAM tables                      | NO           | NO   | NO         |
    -- | CSV        | YES     | CSV storage engine                                         | NO           | NO   | NO         |
    -- | MyISAM     | DEFAULT | Default engine as of MySQL 3.23 with great performance     | NO           | NO   | NO         |
    -- | InnoDB     | YES     | Supports transactions, row-level locking, and foreign keys | YES          | YES  | YES        |
    -- | MEMORY     | YES     | Hash based, stored in memory, useful for temporary tables  | NO           | NO   | NO         |
    -- +------------+---------+------------------------------------------------------------+--------------+------+------------+
    -- 6 rows in set (0.00 sec)

If you see Mroonga storage engine like the above, the installation is well done.

If Mroonga isn't shown, run the following command. If you don't install Mroonga by package, the path of install.sql may be different:

    % mysql -uroot test < /usr/share/mroonga/install.sql

If you can't find install.sql, run INSTALL PLUGIN command like the following manually:

    INSTALL PLUGIN Mroonga SONAME 'ha_mroonga.so';

Then, check server variable whether correct Mroonga version is installed by SHOW VARIABLES command.

    SHOW VARIABLES LIKE 'mroonga_version';

  Mode

Mroonga has the following two modes.

•   storage mode

•   wrapper mode

    With the storage mode, we use Groonga for both of the full text search function and the data storage. Since all functions of storage engine are realised with Groonga, aggregations are fast, that is one of Groonga's advantages, and you can manage the database directly by groonga command.

    The structure of the storage mode is the following. You use it instead of existing storage engines like MyISAM or InnoDB [image: storage mode] [image]

    With the wrapper mode, Groonga is used for full text search function only, and another existing storage engine like InnoDB is used for storing data. By using wrapper mode, you combine InnoDB that is well-use as the storage engine and Mroonga that is a proven full text search engine, and you can use it as the stable database having the fast full text search function.

    The structure of the wrapper mode is the following. Full text search related operations are done by Mroonga, and other operations are done by existing storage engines like MyISAM, InnoDB etc. Mroonga is located between SQL Handler that processes SQL and an existing storage engine, thus all data goes through Mroonga With this way, full text search indexing etc. are done transparently. [image: wrapper mode] [image]

  Storage mode

Here we explain how to use storage mode of Mroonga

  How to use full text search

After confirming the installation, let's create a table. The important point is to specify Mroonga by ENGINE = Mroonga:

    CREATE TABLE diaries (
      id INT PRIMARY KEY AUTO_INCREMENT,
      content VARCHAR(255),
      FULLTEXT INDEX (content)
    ) ENGINE = Mroonga DEFAULT CHARSET utf8;
    -- Query OK, 0 rows affected (0.10 sec)

We put data by INSERT:

    INSERT INTO diaries (content) VALUES ("It'll be fine tomorrow.");
    -- Query OK, 1 row affected (0.01 sec)
    INSERT INTO diaries (content) VALUES ("It'll rain tomorrow");
    -- Query OK, 1 row affected (0.00 sec)

Try full text search:

    SELECT * FROM diaries WHERE MATCH(content) AGAINST("+fine" IN BOOLEAN MODE);
    -- +----+-----------------------------------------+
    -- | id | content                                 |
    -- +----+-----------------------------------------+
    -- |  1 | It'll be fine tomorrow. |
    -- +----+-----------------------------------------+
    -- 1 row in set (0.00 sec)

Yes, full text search works.

  How to get search score

Note: In version 1.0.0 or before, Mroonga used a special column named _score to get search score. From version 1.0.0, it follows MySQL's standard way to get search score.

We often want to display more relevant results first in full text search. We use search score in such case.

We can get search score by MySQL's standard way [1], i.e. we use MATCH...AGAINST in one of columns in SELECT or ORDER BY.

Let's try:

    INSERT INTO diaries (content) VALUES ("It's fine today. It'll be fine tomorrow as well.");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO diaries (content) VALUES ("It's fine today. But it'll rain tomorrow.");
    -- Query OK, 1 row affected (0.00 sec)
    SELECT *,
           MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE)
      FROM diaries
     WHERE MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE)
     ORDER BY
           MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) DESC;
    -- +----+--------------------------------------------------+---------------------------------------------------+
    -- | id | content                                          | MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) |
    -- +----+--------------------------------------------------+---------------------------------------------------+
    -- |  3 | It's fine today. It'll be fine tomorrow as well. |                                                 2 |
    -- |  1 | It'll be fine tomorrow.                          |                                                 1 |
    -- |  4 | It's fine today. But it'll rain tomorrow.        |                                                 1 |
    -- +----+--------------------------------------------------+---------------------------------------------------+
    -- 3 rows in set (0.00 sec)

The result having the search word fine more, i.e. id = 3 message having the higher search score, is displayed first. And you also get search score by using MATCH AGAINST in SELECT phrase.

You can use AS to change the attribute name:

    SELECT *,
           MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) AS score
      FROM diaries
     WHERE MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE)
     ORDER BY
           MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) DESC;
    -- +----+--------------------------------------------------+-------+
    -- | id | content                                          | score |
    -- +----+--------------------------------------------------+-------+
    -- |  3 | It's fine today. It'll be fine tomorrow as well. |     2 |
    -- |  1 | It'll be fine tomorrow.                          |     1 |
    -- |  4 | It's fine today. But it'll rain tomorrow.        |     1 |
    -- +----+--------------------------------------------------+-------+
    -- 3 rows in set (0.00 sec)

  How to specify the parser for full text search

MySQL has the following syntax to specify the parser [2] for full text search:

    FULLTEXT INDEX (content) WITH PARSER parser_name

To use this syntax, you need to register all parsers in MySQL beforehand. On the other hand, Groonga can dynamically add a tokenizer, that is a parser in MySQL. So if use this syntax in Mroonga, tokenizers that are added in Groonga dynamically cannot be supported. We think that this limitation decreases the convenience, and we choose our own syntax using COMMENT like the following:

    FULLTEXT INDEX (content) COMMENT 'tokenizer "TokenMecab"'

Note: COMMENT in FULLTEXT INDEX is only supported MySQL 5.5 or later. If you use MySQL 5.1, use mroonga_default_tokenizer variable described below.

You can specify one of the following values as the tokenizer. "tokenizer" in Mroonga equals to "parser" in MySQL.

  Available tokenizers
Tokenizer | Description
off | (Deprecated) It does not tokenize at all. Use off if you want to treat content as is. For example, this value is used for prefix search.
none | Added in version 8.07: It does not tokenize at all. Use none if you want to treat content as is. For example, this value is used for prefix search.
TokenBigram | It tokenises in bigram. But continuous alphabets, numbers or symbols are treated as a token. So there can exist tokes with 3 letters or more. It is to reduce noises. This is the default value.
TokenMecab | It tokenises using MeCab. Groonga should be built with MeCab support.
TokenBigramSplitSymbol | It tokenises in bigram. Unlike TokenBigram, continuous symbols are not treated as a token, but tokenised in bigram. When you use TokenBigramSplitSymbol instead of TokenBigram, "!?" can match "!?!?!?" in "Is it really!?!?!?". But when you use TokenBigram, only "!?!?!?" can match as well.
TokenBigramSplitSymbolAlpha | It tokenise in bigram. In addition to TokenBigramSplitSymbol, continuous alphabets are not treated as a token either, but tokenised in bigram. When you use TokenBigramSplitSymbolAlpha instead of TokenBigram, "real" can match "Is it really?". But when you use TokenBigram, only "really" can match as well.
TokenBigramSplitSymbolAlphaDigit | It tokenise in bigram. In addition to TokenBigramSplitSymbolAlpha, continuous numbers are not treated as a token either, but tokenised in bigram. So any kind of characters are treated equally in bigram. When you use TokenBigramSplitSymbolAlphaDigit instead of TokenBigram, "567" can match "090-0123-4567". But when you use TokenBigram, only "4567" can match as well.
TokenBigramIgnoreBlank | It tokenise in bigram. Unlike TokenBigram, it ignores white spaces. When you use TokenBigramIgnoreBlank instead of TokenBigram, "みなさん" can match "み な さ ん 注 目". But when you use TokenBigram, only "み な さ ん" can match as well.
TokenBigramIgnoreBlankSplitSymbol | It tokenise in bigram. Unlike TokenBigramSplitSymbol, it ignores white spaces. When you use TokenBigramIgnoreBlankSplitSymbol instead of TokenBigramSplitSymbol, "???" can match "! ? ???". But when you use TokenBigramSplitSymbol, only "? ??" can match as well.
TokenBigramIgnoreBlankSplitSymbolAlpha | It tokenise in bigram. Unlike TokenBigramSplitSymbolAlpha, it ignores white spaces. When you use TokenBigramIgnoreBlankSplitSymbolAlpha instead of TokenBigramSplitSymbolAlpha, "ama" can match "I am a pen.". But when you use TokenBigramSplitSymbolAlpha, only "am a" can match as well.
TokenBigramIgnoreBlankSplitSymbolAlphaDigit | It tokenise in bigram. Unlike TokenBigramSplitSymbolAlphaDigit, it ignores white spaces. When you use TokenBigramIgnoreBlankSplitSymbolAlphaDigit instead of TokenBigramSplitSymbolAlphaDigit, "9001" can match "090 0123 4567". But when you use TokenBigramSplitSymbolAlphaDigit, only "90 01" can match as well.
TokenDelimit | It tokenise by splitting with a white space. "movie horror topic" will be tokenised as "movie", "horror", "topic".
TokenDelimitNull | It tokenise by splitting with a null character (\\0). "movie\\0horror\\0topic" will be tokenised as "movie", "horror", "topic".
TokenUnigram | It tokenises in unigram. But continuous alphabets, numbers or symbols are treated as a token. So there can exist tokes with 2 letters or more. It is to reduce noises.
TokenTrigram | It tokenises in trigram. But continuous alphabets, numbers or symbols are treated as a token. So there can exist tokes with 4 letters or more. It is to reduce noises.

You can specify the default parser by passing --with-default-tokenizer option in configure when you build Mroonga:

    ./configure --with-default-tokenizer TokenMecab ...

Or you can set mroonga_default_tokenizer variable in my.cnf or by SQL. If you specify it in my.cnf, the change will not be lost after restarting MySQL, but you need to restart MySQL to make it effective. On the other hand, if you set it in SQL, the change is effective immediately, but it will be lost when you restart MySQL.

my.cnf:

    [mysqld]
    mroonga_default_tokenizer=TokenMecab

SQL:

    SET GLOBAL mroonga_default_tokenizer = TokenMecab;
    -- Query OK, 0 rows affected (0.00 sec)

  How to specify the normalizer

Mroonga uses normalizer corresponding to the encoding of document. It is used when tokenizing text and storing table key.

It is used NormalizerMySQLGeneralCI normalizer when the encoding is utf8_general_ci or utf8mb4_general_ci.

It is used NormalizerMySQLUnicodeCI normalizer when the encoding is utf8_unicode_ci or utf8mb4_unicode_ci.

It isn't used normalizer when the encoding is utf8_bin.

Here is an example that uses NormalizerMySQLUnicodeCI normalizer by specifying utf8_unicode_ci:

    SET NAMES utf8;
    -- Query OK, 0 rows affected (0.00 sec)
    CREATE TABLE diaries (
      day DATE PRIMARY KEY,
      content VARCHAR(64) NOT NULL,
      FULLTEXT INDEX (content)
    ) Engine=Mroonga DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
    -- Query OK, 0 rows affected (0.18 sec)
    INSERT INTO diaries VALUES ("2013-04-23", "ブラックコーヒーを飲んだ。");
    -- Query OK, 1 row affected (0.00 sec)
    SELECT *
      FROM diaries
     WHERE MATCH (content) AGAINST ("+ふらつく" IN BOOLEAN MODE);
    -- +------------+-----------------------------------------+
    -- | day        | content                                 |
    -- +------------+-----------------------------------------+
    -- | 2013-04-23 | ブラックコーヒーを飲んだ。 |
    -- +------------+-----------------------------------------+
    -- 1 row in set (0.00 sec)
    SELECT *
      FROM diaries
     WHERE MATCH (content) AGAINST ("+ﾌﾞﾗｯｸ" IN BOOLEAN MODE);
    -- +------------+-----------------------------------------+
    -- | day        | content                                 |
    -- +------------+-----------------------------------------+
    -- | 2013-04-23 | ブラックコーヒーを飲んだ。 |
    -- +------------+-----------------------------------------+
    -- 1 row in set (0.00 sec)

Mroonga has the following syntax to specify Groonga's normalizer:

    FULLTEXT INDEX (content) COMMENT 'normalizer "NormalizerAuto"'

See Groonga's documentation about normalizer <https://groonga.org/docs/reference/normalizers.html> for more details.

Here is an example that uses NormalizerAuto normalizer:

    SET NAMES utf8;
    -- Query OK, 0 rows affected (0.00 sec)
    CREATE TABLE diaries (
      day DATE PRIMARY KEY,
      content VARCHAR(64) NOT NULL,
      FULLTEXT INDEX (content) COMMENT 'normalizer "NormalizerAuto"'
    ) Engine=Mroonga DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
    -- Query OK, 0 rows affected (0.19 sec)
    INSERT INTO diaries VALUES ("2013-04-23", "ブラックコーヒーを飲んだ。");
    -- Query OK, 1 row affected (0.00 sec)
    SELECT *
      FROM diaries
     WHERE MATCH (content) AGAINST ("+ふらつく" IN BOOLEAN MODE);
    -- Empty set (0.00 sec)
    SELECT *
      FROM diaries
     WHERE MATCH (content) AGAINST ("+ﾌﾞﾗｯｸ" IN BOOLEAN MODE);
    -- +------------+-----------------------------------------+
    -- | day        | content                                 |
    -- +------------+-----------------------------------------+
    -- | 2013-04-23 | ブラックコーヒーを飲んだ。 |
    -- +------------+-----------------------------------------+
    -- 1 row in set (0.00 sec)

  How to specify the token filters

Mroonga has the following syntax to specify Groonga's token filters.:

    FULLTEXT INDEX (content) COMMENT 'token_filters "TokenFilterStem"'

Here is an example that uses TokenFilterStem token filter:

    SELECT mroonga_command('register token_filters/stem');
    -- +------------------------------------------------+
    -- | mroonga_command('register token_filters/stem') |
    -- +------------------------------------------------+
    -- | true                                           |
    -- +------------------------------------------------+
    -- 1 row in set (0.00 sec)
    CREATE TABLE memos (
      id INT NOT NULL PRIMARY KEY,
      content TEXT NOT NULL,
      FULLTEXT INDEX (content) COMMENT 'normalizer "NormalizerAuto", token_filters "TokenFilterStem"'
    ) Engine=Mroonga DEFAULT CHARSET=utf8;
    -- Query OK, 0 rows affected (0.18 sec)
    INSERT INTO memos VALUES (1, "I develop Groonga");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (2, "I'm developing Groonga");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (3, "I developed Groonga");
    -- Query OK, 1 row affected (0.00 sec)
    SELECT *
      FROM memos
     WHERE MATCH (content) AGAINST ("+develops" IN BOOLEAN MODE);
    -- +----+------------------------+
    -- | id | content                |
    -- +----+------------------------+
    -- |  1 | I develop Groonga      |
    -- |  2 | I'm developing Groonga |
    -- |  3 | I developed Groonga    |
    -- +----+------------------------+
    -- 3 rows in set (0.01 sec)

See Groonga's documentation about token filter documentation <https://groonga.org/docs/reference/token_filters.html> for more details.

Here is an example that uses TokenFilterStopWord token filter:

    SELECT mroonga_command("register token_filters/stop_word");
    -- +-----------------------------------------------------+
    -- | mroonga_command("register token_filters/stop_word") |
    -- +-----------------------------------------------------+
    -- | true                                                |
    -- +-----------------------------------------------------+
    -- 1 row in set (0.00 sec)
    CREATE TABLE terms (
      term VARCHAR(64) NOT NULL PRIMARY KEY,
      is_stop_word BOOL NOT NULL
    ) Engine=Mroonga COMMENT='tokenizer "TokenBigram", token_filters "TokenFilterStopWord"' DEFAULT CHARSET=utf8;
    -- Query OK, 0 rows affected (0.12 sec)
    CREATE TABLE memos (
      id INT NOT NULL PRIMARY KEY,
      content TEXT NOT NULL,
      FULLTEXT INDEX (content) COMMENT 'table "terms"'
    ) Engine=Mroonga DEFAULT CHARSET=utf8;
    -- Query OK, 0 rows affected (0.17 sec)
    INSERT INTO terms VALUES ("and", true);
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (1, "Hello");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (2, "Hello and Good-bye");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (3, "Good-bye");
    -- Query OK, 1 row affected (0.00 sec)
    SELECT *
      FROM memos
     WHERE MATCH (content) AGAINST ('+"Hello and"' IN BOOLEAN MODE);
    -- +----+--------------------+
    -- | id | content            |
    -- +----+--------------------+
    -- |  1 | Hello              |
    -- |  2 | Hello and Good-bye |
    -- +----+--------------------+
    -- 2 rows in set (0.01 sec)

It's used that specifying the lexicon table for fulltext search.

  How to specify Groonga's column flags

Mroonga has the following syntax to specify Groonga's column flags:

    content TEXT COMMENT 'flags "COLUMN_SCALAR|COMPRESS_ZLIB"'

Here is an example that uses COMPRESS_ZLIB flag:

    CREATE TABLE entries (
      id INT UNSIGNED PRIMARY KEY,
      content TEXT COMMENT 'flags "COLUMN_SCALAR|COMPRESS_ZLIB"'
    ) Engine=Mroonga DEFAULT CHARSET=utf8;
    -- Query OK, 0 rows affected (0.12 sec)

See Groonga's documentation about column flags <https://groonga.org/docs/reference/commands/column_create.html#parameters> for more details.

  How to use geolocation search

In storage mode, you can use fast geolocation search in addition to full text search. But unlike MyISAM, you can only store POINT type data. You cannot store other types data like LINE. And fast search using index only supports MBRContains. It does not support MBRDisjoint.

For the table definition for geolocation search, you need to define a POINT type column like in MyISAM and define SPATIAL INDEX for it.:

    CREATE TABLE shops (
      id INT PRIMARY KEY AUTO_INCREMENT,
      name VARCHAR(255),
      location POINT NOT NULL,
      SPATIAL INDEX (location)
    ) ENGINE = Mroonga;
    -- Query OK, 0 rows affected (0.06 sec)

To store data, you create POINT type data by using geomFromText() function like in MyISAM:

    INSERT INTO shops VALUES (null, 'Nezu''s Taiyaki', GeomFromText('POINT(139.762573 35.720253)'));
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO shops VALUES (null, 'Naniwaya', GeomFromText('POINT(139.796234 35.730061)'));
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO shops VALUES (null, 'Yanagiya Taiyaki', GeomFromText('POINT(139.783981 35.685341)'));
    -- Query OK, 1 row affected (0.00 sec)

If you want to find shops within the rectangle where Ikebukuro station (139.7101 35.7292) is the top-left point and Tokyo Station (139.7662 35.6815) is the bottom-right point, SELECT phrase is like the following:

    SELECT id, name, AsText(location)
      FROM shops
     WHERE MBRContains(GeomFromText('LineString(139.7101 35.7292, 139.7662 35.6815)'), location);
    -- +----+-----------------------+------------------------------------------+
    -- | id | name                  | AsText(location)                         |
    -- +----+-----------------------+------------------------------------------+
    -- |  1 | Nezu's Taiyaki        | POINT(139.762572777778 35.7202527777778) |
    -- +----+-----------------------+------------------------------------------+
    -- 1 row in set (0.00 sec)

Here you can search by geolocation!

  How to get the record ID

Groonga assigns a unique number to identify the record when a record is added in the table.

To make the development of applications easier, you can get this record ID by SQL in Mroonga

To get the record ID, you need to create a column named _id when you create a table:

    CREATE TABLE memos (
      _id INT,
      content VARCHAR(255),
      UNIQUE KEY (_id) USING HASH
    ) ENGINE = Mroonga;
    -- Query OK, 0 rows affected (0.04 sec)

Tye typo of _id column should be integer one (TINYINT, SMALLINT, MEDIUMINT, INT or BIGINT).

You can create an index for _id column, but it should be HASH type.

Let's add records in the table by INSERT. Since _id column is implemented as a virtual column and its value is assigned by Groonga, you cannot specify the value when updating. So you need to exclude it from setting columns, or you need to use null as its value:

    INSERT INTO memos VALUES (null, "Saury for today's dinner.");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (null, "Update mroonga tomorrow.");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (null, "Buy some dumpling on the way home.");
    -- Query OK, 1 row affected (0.00 sec)
    INSERT INTO memos VALUES (null, "Thank God It's meat day.");
    -- Query OK, 1 row affected (0.00 sec)

To get the record ID, you invoke SELECT with _id column:

    SELECT * FROM memos;
    -- +------+------------------------------------------+
    -- | _id  | content                                  |
    -- +------+------------------------------------------+
    -- |    1 | Saury for today's dinner.                |
    -- |    2 | Update mroonga tomorrow.                 |
    -- |    3 | Buy some dumpling on the way home.       |
    -- |    4 | Thank God It's meat day.                 |
    -- +------+------------------------------------------+
    -- 4 rows in set (0.00 sec)

By using last_insert_grn_id function, you can also get the record ID that is assigned by the last INSERT:

    INSERT INTO memos VALUES (null, "Just one bottle of milk in the fridge.");
    -- Query OK, 1 row affected (0.00 sec)
    SELECT last_insert_grn_id();
    -- +----------------------+
    -- | last_insert_grn_id() |
    -- +----------------------+
    -- |                    5 |
    -- +----------------------+
    -- 1 row in set (0.00 sec)

last_insert_grn_id function is included in Mroonga as a User-Defined Function (UDF), but if you have not yet register it in MySQL by CREATE FUNCTION, you need to invoke the following SQL for defining a function:

    CREATE FUNCTION last_insert_grn_id RETURNS INTEGER SONAME 'ha_mroonga.so';

As you can see in the example above, you can get the record ID by _id column or last_insert_grn_id function. It will be useful to use this value in the ensuing SQL queries like UPDATE:

    UPDATE memos SET content = "So much milk in the fridge." WHERE _id = last_insert_grn_id();
    -- Query OK, 1 row affected (0.00 sec)
    -- Rows matched: 1  Changed: 1  Warnings: 0

  How to get snippet (Keyword in context)

Mroonga provides functionality to get keyword in context. It is implemented as mroonga_snippet() <> UDF.

  How to use similar search

Similar search is supported by specifying document itself instead of specifying keywords in query.

It is useful if you want to find documents which is related to specific document.

Here is the schema definition for execution examples:

    CREATE TABLE similarities (
      id INT PRIMARY KEY AUTO_INCREMENT,
      title VARCHAR(32),
      content VARCHAR(255),
      FULLTEXT INDEX (content)
    ) ENGINE = Mroonga DEFAULT CHARSET utf8;

Here is the sample data for execution examples:

    INSERT INTO similarities (title, content)
           VALUES ('Groonga similar search', 'Groonga is an open-source fulltext search engine and column store.');
    INSERT INTO similarities (title, content)
           VALUES ('Mroonga similar search', 'Mroonga is an open-source storage engine for fast fulltext search with MySQL.');
    INSERT INTO similarities (title, content)
           VALUES ('Rroonga library', 'A library to use Groonga features from Ruby.');

Here is the example of similar search by content itself.

    SELECT title FROM similarities WHERE MATCH(content) AGAINST ('There are many open-source fulltext search engine.' IN NATURAL LANGUAGE MODE);

Note that you need to specify an content of document in AGAINST('...' IN NATURAL LANGUAGE MODE).

    SELECT title FROM similarities WHERE MATCH(content) AGAINST ('There are many open-source fulltext search engine.' IN NATURAL LANGUAGE MODE);
    -- +------------------------+
    -- | title                  |
    -- +------------------------+
    -- | Groonga similar search |
    -- | Mroonga similar search |
    -- +------------------------+
    -- 2 rows in set (0.00 sec)

To improve degree of similarity, you may need to use language specific tokenizer or use labeled data such as tag to get rid of some undesirable search results.

For example, if you want to execute similar search against Japanese text, it is recommended to use language specific tokenizer - TokenMecab. Here is the schema definition to use TokenMecab tokenizer:

    CREATE TABLE similarities (
      id INT PRIMARY KEY AUTO_INCREMENT,
      title VARCHAR(32),
      content VARCHAR(255),
      FULLTEXT INDEX (content) COMMENT 'tokenizer "TokenMecab"'
    ) ENGINE = Mroonga DEFAULT CHARSET utf8;

  How to run Groonga command

In storage mode, Mroonga stores all your data into Groonga database. You can access Groonga database by SQL with Mroonga. SQL is very powerful but it is not good for some operations such as faceted search.

Faceted search is popular recently. Many online shopping sites such as amazon.com and ebay.com support faceted search. Faceted search refines the current search by available search parameters before users refine their search. And faceted search shows refined searches. Users just select a refined search. Users benefit from faceted search:

•   Users don't need to think about how to refine their search. Users just select a showed refined search.

•   Users don't get boared "not match" page. Faceted search showes only refined searches that has one or more matched items.

    Faceted search needs multiple GROUP BY operations against searched result set. To do faceted search by SQL, multiple SELECT requests are needed. It is not effective.

    Groonga can do faceted search by only one groonga command. It is effective. Groonga has the select command that can search records with faceted search. Faceted search is called as "drilldown" in Groonga. See Groonga's documentation about select command <https://groonga.org/docs/reference/commands/select.html> for more details.

    Mroonga provides mroonga_command() function. You can run Groonga command in SQL by the function. But you should use only select command. Other commands that change schema or data may break consistency.

    Here is the schema definition for execution examples:

        CREATE TABLE diaries (
          id INT PRIMARY KEY AUTO_INCREMENT,
          content VARCHAR(255),
          date DATE,
          year YEAR,
          `year_month` VARCHAR(9),
          tag VARCHAR(32),
          FULLTEXT INDEX (content)
        ) ENGINE = Mroonga DEFAULT CHARSET utf8;

    Here is the sample data for execution examples:

        INSERT INTO diaries (content, date, year, `year_month`, tag)
               VALUES ('Groonga is an open-source fulltext search engine and column store.',
                       '2013-04-08',
                       '2013',
                       '2013-04',
                       'groonga');
        INSERT INTO diaries (content, date, year, `year_month`, tag)
               VALUES ('Mroonga is an open-source storage engine for fast fulltext search with MySQL.',
                       '2013-04-09',
                       '2013',
                       '2013-04',
                       'MySQL');
        INSERT INTO diaries (content, date, year, `year_month`, tag)
               VALUES ('Tritonn is a patched version of MySQL that supports better fulltext search function with Senna.',
                       '2013-03-29',
                       '2013',
                       '2013-03',
                       'MySQL');

    Each record has groonga or MySQL as tag. Each record also has year and year_month. You can use tag, year and year_month as faceted search keys.

    Groonga calls faceted search as drilldown. So parameter key in Groonga is --drilldown. Groonga returns search result as JSON. So mroonga_command() also returns search result as JSON. It is not SQL friendly. You need to parse search result JSON by yourself.

    Here is the example of faceted search by all available faceted search keys (result JSON is pretty printed):

        SELECT mroonga_command("select diaries --output_columns _id --limit 0 --drilldown tag,year,year_month") AS faceted_result;
        -- +-----------------------------+
        -- | faceted_result              |
        -- +-----------------------------+
        -- | [[[3],                      |
        -- |   [["_id","UInt32"]]],      |
        -- |  [[2],                      |
        -- |   [["_key","ShortText"],    |
        -- |    ["_nsubrecs","Int32"]],  |
        -- |   ["groonga",1],            |
        -- |   ["MySQL",2]],             |
        -- |  [[1],                      |
        -- |   [["_key","Time"],         |
        -- |    ["_nsubrecs","Int32"]],  |
        -- |   [1356998400.0,3]],        |
        -- |  [[2],                      |
        -- |   [["_key","ShortText"],    |
        -- |    ["_nsubrecs","Int32"]],  |
        -- |   ["2013-04",2],            |
        -- |   ["2013-03",1]]]           |
        -- +-----------------------------+
        -- 1 row in set (0.00 sec)

    The first element [[3], [["_id","UInt32"]]] is normal search result. It's not faceted search result.

    The second, third and forth elements are faceted search results:

        [[2],
         [["_key","ShortText"],
          ["_nsubrecs","Int32"]],
         ["groonga",1],
         ["MySQL",2]]

        [[1],
         [["_key","Time"],
          ["_nsubrecs","Int32"]],
         [1356998400.0,3]]

        [[2],
         [["_key","ShortText"],
          ["_nsubrecs","Int32"]],
         ["2013-04",2],
         ["2013-03",1]]

    The order of faceted search results is corresponding to the value of --drilldown. In this example, we specified tag, year and year_month as --drilldown value. So the first faceted search result is for tag, the second one is for year and the third one is for year_month.

    Each faceted search result uses the following format. This is the same as normal search result:

        [[${THE_NUMBER_OF_RECORDS}],
         [[${OUTPUT_COLUMN_NAME_0}, ${OUTPUT_COLUMN_TYPE_0}],
          [${OUTPUT_COLUMN_NAME_1}, ${OUTPUT_COLUMN_TYPE_1}],
          ...,
          [${OUTPUT_COLUMN_NAME_N}, ${OUTPUT_COLUMN_TYPE_N}]]
         [${OUTPUT_COLUMN_VALUE_0_FOR_RECORD_0},
          ${OUTPUT_COLUMN_VALUE_1_FOR_RECORD_0},
          ...,
          ${OUTPUT_COLUMN_VALUE_N_FOR_RECORD_0}],
         [${OUTPUT_COLUMN_VALUE_0_FOR_RECORD_1},
          ${OUTPUT_COLUMN_VALUE_1_FOR_RECORD_1},
          ...,
          ${OUTPUT_COLUMN_VALUE_N_FOR_RECORD_1}],
         ...
         [${OUTPUT_COLUMN_VALUE_0_FOR_RECORD_M},
          ${OUTPUT_COLUMN_VALUE_1_FOR_RECORD_M},
          ...,
          ${OUTPUT_COLUMN_VALUE_N_FOR_RECORD_M}]]

    The _key column value in faceted search result shows faceted search key.

    For example, the first faceted search result (the faceted search result for tag) shows that matched records (all records in this case) have groonga and MySQL as tag value:

        [[2],
         [["_key","ShortText"],
          ["_nsubrecs","Int32"]],
         ["groonga",1],
         ["MySQL",2]]

    The _nsubrecs column value in faceted search result shows the number of records that have the corresponding faceted search key.

    For example, the first faceted search result (the faceted search result for tag) shows that there are 1 record that has groonga as tag value and 2 records that have MySQL as tag value:

        [[2],
         [["_key","ShortText"],
          ["_nsubrecs","Int32"]],
         ["groonga",1],
         ["MySQL",2]]

    See Groonga's documentation about select command <https://groonga.org/docs/reference/commands/select.html> for more details.

  How to search by regular expression

In storage mode, you can use a Groonga's functionality from Mroonga as described above. Thus, you can search records by using a regular expression via Groonga's functionality.

There are some conditions to use regular expression in Mroonga.

•   Create an index with TokenRegexp tokenizer which is used in COMMENT

•   Use *SS pragma and @~ in WHERE MATCH ... AGAINST clause

    Here is the example of search by regular expression.

        CREATE TABLE paths (
          content text,
          FULLTEXT INDEX content_index (content) COMMENT 'tokenizer "TokenRegexp", normalizer "NormalizerAuto"'
        ) ENGINE=Mroonga DEFAULT CHARSET=utf8mb4;
        INSERT INTO paths VALUES ('/usr/bin/groonga');
        INSERT INTO paths VALUES ('/var/log/auth.log');
        INSERT INTO paths VALUES ('/var/log/messages');
        INSERT INTO paths VALUES ('/tmp/local/var/log/auth.log');
        SELECT * FROM paths WHERE MATCH(content) AGAINST ('*SS content @~ "\\\\A/var/log/auth"' IN BOOLEAN MODE);
        -- +-------------------+
        -- | content           |
        -- +-------------------+
        -- | /var/log/auth.log |
        -- +-------------------+
        -- 1 row in set (0.024 sec)

    By using *SS <#boolean-mode-pragma-ss> pragma, you can search the records which matches /var/log/auth.log with content @~ "\\\\A/var/log/auth". @~ is a Groonga's operator which executes a regular expression search, and "\\\\A/var/log/auth" executes prefix search, so it matches to only /var/log/auth.log. /tmp/local/var/log/auth.log doesn't match because it doesn't begin with "/var/log/auth".

    See Groonga's regular expression document <https://groonga.org/docs/reference/regular_expression.html#syntax> for more syntax details.

  Logging

Mroonga outputs the logs by default.

Log files are located in MySQL's data directory with the filename groonga.log.

Here is the example of the log.

    2010-10-07 17:32:39.209379|n|b1858f80|mroonga 1.10 started.
    2010-10-07 17:32:44.934048|d|46953940|hash get not found (key=test)
    2010-10-07 17:32:44.936113|d|46953940|hash put (key=test)

The default log level is NOTICE, i.e. we have important information only and we don't have debug information etc.).

You can get the log level by mroonga_log_level system variable, that is a global variable. You can also modify it dynamically by using SET phrase:

    SHOW VARIABLES LIKE 'mroonga_log_level';
    -- +-------------------+--------+
    -- | Variable_name     | Value  |
    -- +-------------------+--------+
    -- | mroonga_log_level | NOTICE |
    -- +-------------------+--------+
    -- 1 row in set (0.00 sec)
    SET GLOBAL mroonga_log_level=DUMP;
    -- Query OK, 0 rows affected (0.00 sec)
    SHOW VARIABLES LIKE 'mroonga_log_level';
    -- +-------------------+-------+
    -- | Variable_name     | Value |
    -- +-------------------+-------+
    -- | mroonga_log_level | DUMP  |
    -- +-------------------+-------+
    -- 1 row in set (0.00 sec)

Available log levels are the followings.

•   NONE

•   EMERG

•   ALERT

•   CRIT

•   ERROR

•   WARNING

•   NOTICE

•   INFO

•   DEBUG

•   DUMP

    See mroonga_log_level <#server-variable-mroonga-log-level> about details.

    You can reopen the log file by FLUSH LOGS. If you want to rotate the log file without stopping MySQL server, you can do in the following procedure.

1.  change the file name of groonga.log (by using OS's mv command etc.).

2.  invoke "FLUSH LOGS" in MySQL server (by mysql command or mysqladmin command).

  Next step

Now, you can use Mroonga as storage mode! If you want Mroonga to be faster, see also Optimizations <>.

FOOTNOTES

[1] MySQL 5.1 Reference Manual :: 11 Functions and Operations :: 11.7 Full-Text Search Functions <http://dev.mysql.com/doc/refman/5.1/ja/fulltext-search.html>

[2] In Groonga, we call it a 'tokenizer'.

  Wrapper mode

Here we explain how to use wrapper mode of Mroonga

Changed in version 12.02: Dropped support wrapper mode with MySQL 8.0 or later.

  How to use wrapper mode

In wrapper mode, Mroonga works in wrapping an existing storage engine. To specify the wrapped storage engine, we use SQL comment like COMMENT = 'engine "InnoDB"' for now.

Note: For now, a primary key is mandatory in wrapper mode. That is not the case with storage mode.

Note: Wrapper mode supports the followings, that are not supported in storage mode for now.

•   null value

•   transaction (if storage engine supports. Note that rollback causes mismatch of indexes, it may affects search results, so recreate index of Mroonga in such a case.)

  How to use full text search

After confirming the installation, let's create a table. The important point is to specify Mroonga by ENGINE = Mroonga.

    mysql> CREATE TABLE diaries (
        ->   id INT PRIMARY KEY AUTO_INCREMENT,
        ->   content VARCHAR(255),
        ->   FULLTEXT INDEX (content)
        -> ) ENGINE = Mroonga COMMENT = 'engine "InnoDB"' DEFAULT CHARSET utf8;
    Query OK, 0 rows affected (0.52 sec)

We put data by INSERT.

    mysql> INSERT INTO diaries (content) VALUES ("It'll be fine tomorrow.");
    Query OK, 1 row affected (0.01 sec)
    mysql> INSERT INTO diaries (content) VALUES ("It'll rain tomorrow");
    Query OK, 1 row affected (0.00 sec)

Try full text search.

    mysql> SELECT * FROM diaries WHERE MATCH(content) AGAINST("+fine" IN BOOLEAN MODE);
    +----+-----------------------------------------+
    | id | content                                 |
    +----+-----------------------------------------+
    |  1 | It'll be fine tomorrow. |
    +----+-----------------------------------------+
    1 row in set (0.00 sec)

Yes, full text search works.

  How to get search score

We often want to display more relevant results first in full text search. We use search score in such case.

We can get search score by MySQL's standard way [1], i.e. we use MATCH...AGAINST in one of columns in SELECT or ORDER BY.

Let's try.

    mysql> INSERT INTO diaries (content) VALUES ("It's fine today. It'll be fine tomorrow as well.");
    Query OK, 1 row affected (0.00 sec)
    mysql> INSERT INTO diaries (content) VALUES ("It's fine today. But it'll rain tomorrow.");
    Query OK, 1 row affected (0.00 sec)
    mysql> SELECT *, MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) FROM diaries WHERE MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) ORDER BY MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) DESC;
    +----+--------------------------------------------------+---------------------------------------------------+
    | id | content                                          | MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) |
    +----+--------------------------------------------------+---------------------------------------------------+
    |  3 | It's fine today. It'll be fine tomorrow as well. |                                                 2 |
    |  1 | It'll be fine tomorrow.                          |                                                 1 |
    |  4 | It's fine today. But it'll rain tomorrow.        |                                                 1 |
    +----+--------------------------------------------------+---------------------------------------------------+
    3 rows in set (0.00 sec)

The result having the search word fine more, i.e. id = 3 message having the higher search score, is displayed first. And you also get search score by using MATCH AGAINST in SELECT phrase.

You can use AS to change the attribute name.

    mysql> SELECT *, MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) AS score FROM diaries WHERE MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) ORDER BY MATCH (content) AGAINST ("+fine" IN BOOLEAN MODE) DESC;
    +----+--------------------------------------------------+-------+
    | id | content                                          | score |
    +----+--------------------------------------------------+-------+
    |  3 | It's fine today. It'll be fine tomorrow as well. |     2 |
    |  1 | It'll be fine tomorrow.                          |     1 |
    |  4 | It's fine today. But it'll rain tomorrow.        |     1 |
    +----+--------------------------------------------------+-------+
    3 rows in set (0.00 sec)

  How to specify the parser for full text search

MySQL has the following syntax to specify the parser [2] for full text search.

    FULLTEXT INDEX (content) WITH PARSER parser_name

To use this syntax, you need to register all parsers in MySQL beforehand. On the other hand, Groonga can dynamically add a tokenizer, that is a parser in MySQL. So if use this syntax in Mroonga, tokenizers that are added in Groonga dynamically cannot be supported. We think that this limitation decreases the convenience, and we choose our own syntax using COMMENT like the following.

    FULLTEXT INDEX (content) COMMENT 'tokenizer "TokenMecab"'

Note: COMMENT in FULLTEXT INDEX is only supported MySQL 5.5 or later. If you use MySQL 5.1, use mroonga_default_parser variable described below.

You can specify one of following values as the tokenizer.

  Available tokenizers
Tokenizer | Description
off | (Deprecated) It does not tokenize at all. Use off if you want to treat content as is. For example, this value is used for prefix search.
none | Added in version 8.07: It does not tokenize at all. Use none if you want to treat content as is. For example, this value is used for prefix search.
TokenBigram | It tokenises in bigram. But continuous alphabets, numbers or symbols are treated as a token. So there can exist tokes with 3 letters or more. It is to reduce noises. This is the default value.
TokenMecab | It tokenises using MeCab. Groonga should be built with MeCab support.
TokenBigramSplitSymbol | It tokenises in bigram. Unlike TokenBigram, continuous symbols are not treated as a token, but tokenised in bigram. When you use TokenBigramSplitSymbol instead of TokenBigram, "!?" can match "!?!?!?" in "Is it really!?!?!?". But when you use TokenBigram, only "!?!?!?" can match as well.
TokenBigramSplitSymbolAlpha | It tokenise in bigram. In addition to TokenBigramSplitSymbol, continuous alphabets are not treated as a token either, but tokenised in bigram. When you use TokenBigramSplitSymbolAlpha instead of TokenBigram, "real" can match "Is it really?". But when you use TokenBigram, only "really" can match as well.
TokenBigramSplitSymbolAlphaDigit | It tokenise in bigram. In addition to TokenBigramSplitSymbolAlpha, continuous numbers are not treated as a token either, but tokenised in bigram. So any kind of characters are treated equally in bigram. When you use TokenBigramSplitSymbolAlphaDigit instead of TokenBigram, "567" can match "090-0123-4567". But when you use TokenBigram, only "4567" can match as well.
TokenBigramIgnoreBlank | It tokenise in bigram. Unlike TokenBigram, it ignores white spaces. When you use TokenBigramIgnoreBlank instead of TokenBigram, "みなさん" can match "み な さ ん 注 目". But when you use TokenBigram, only "み な さ ん" can match as well.
TokenBigramIgnoreBlankSplitSymbol | It tokenise in bigram. Unlike TokenBigramSplitSymbol, it ignores white spaces. When you use TokenBigramIgnoreBlankSplitSymbol instead of TokenBigramSplitSymbol, "???" can match "! ? ???". But when you use TokenBigramSplitSymbol, only "? ??" can match as well.
TokenBigramIgnoreBlankSplitSymbolAlpha | It tokenise in bigram. Unlike TokenBigramSplitSymbolAlpha, it ignores white spaces. When you use TokenBigramIgnoreBlankSplitSymbolAlpha instead of TokenBigramSplitSymbolAlpha, "ama" can match "I am a pen.". But when you use TokenBigramSplitSymbolAlpha, only "am a" can match as well.
TokenBigramIgnoreBlankSplitSymbolAlphaDigit | It tokenise in bigram. Unlike TokenBigramSplitSymbolAlphaDigit, it ignores white spaces. When you use TokenBigramIgnoreBlankSplitSymbolAlphaDigit instead of TokenBigramSplitSymbolAlphaDigit, "9001" can match "090 0123 4567". But when you use TokenBigramSplitSymbolAlphaDigit, only "90 01" can match as well.
TokenDelimit | It tokenise by splitting with a white space. "movie horror topic" will be tokenised as "movie", "horror", "topic".
TokenDelimitNull | It tokenise by splitting with a null character (\\0). "movie\\0horror\\0topic" will be tokenised as "movie", "horror", "topic".
TokenUnigram | It tokenises in unigram. But continuous alphabets, numbers or symbols are treated as a token. So there can exist tokes with 2 letters or more. It is to reduce noises.
TokenTrigram | It tokenises in trigram. But continuous alphabets, numbers or symbols are treated as a token. So there can exist tokes with 4 letters or more. It is to reduce noises.

You can specify the default tokenizer by passing --with-default-tokenizer option in configure when you build Mroonga.

    ./configure --with-default-tokenizer TokenMecab ...

Or you can set mroonga_default_tokenizer variable in my.cnf or by SQL. If you specify it in my.cnf, the change will not be lost after restarting MySQL, but you need to restart MySQL to make it effective. On the other hand, if you set it in SQL, the change is effective immediately, but it will be lost when you restart MySQL.

my.cnf:

    [mysqld]
    mroonga_default_tokenizer=TokenMecab

SQL:

    mysql> SET GLOBAL mroonga_default_tokenizer = TokenMecab;
    Query OK, 0 rows affected (0.00 sec)

  How to specify the normalizer

Mroonga uses normalizer corresponding to the encoding of document. It is used when tokenizing text and storing table key.

It is used NormalizerMySQLGeneralCI normalizer when the encoding is utf8_general_ci or utf8mb4_general_ci.

It is used NormalizerMySQLUnicodeCI normalizer when the encoding is utf8_unicode_ci or utf8mb4_unicode_ci.

It isn't used normalizer when the encoding is utf8_bin.

Here is an example that uses NormalizerMySQLUnicodeCI normalizer by specifying utf8_unicode_ci.:

    mysql> SET NAMES utf8;
    Query OK, 0 rows affected (0.00 sec)
    mysql> CREATE TABLE diaries (
        ->   day DATE PRIMARY KEY,
        ->   content VARCHAR(64) NOT NULL,
        ->   FULLTEXT INDEX (content)
        -> ) Engine=Mroonga DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
    Query OK, 0 rows affected (0.18 sec)
    mysql> INSERT INTO diaries VALUES ("2013-04-23", "ブラックコーヒーを飲んだ。");
    Query OK, 1 row affected (0.00 sec)
    mysql> SELECT * FROM diaries
        ->        WHERE MATCH (content) AGAINST ("+ふらつく" IN BOOLEAN MODE);
    +------------+-----------------------------------------+
    | day        | content                                 |
    +------------+-----------------------------------------+
    | 2013-04-23 | ブラックコーヒーを飲んだ。 |
    +------------+-----------------------------------------+
    1 row in set (0.00 sec)
    mysql> SELECT * FROM diaries
        ->        WHERE MATCH (content) AGAINST ("+ﾌﾞﾗｯｸ" IN BOOLEAN MODE);
    +------------+-----------------------------------------+
    | day        | content                                 |
    +------------+-----------------------------------------+
    | 2013-04-23 | ブラックコーヒーを飲んだ。 |
    +------------+-----------------------------------------+
    1 row in set (0.00 sec)

Mroonga has the following syntax to specify Groonga's normalizer:

    FULLTEXT INDEX (content) COMMENT 'normalizer "NormalizerAuto"'

See Groonga's documentation about normalizer <https://groonga.org/docs/reference/normalizers.html> for more details.

Here is an example that uses NormalizerAuto normalizer:

    mysql> SET NAMES utf8;
    Query OK, 0 rows affected (0.00 sec)
    mysql> CREATE TABLE diaries (
        ->   day DATE PRIMARY KEY,
        ->   content VARCHAR(64) NOT NULL,
        ->   FULLTEXT INDEX (content) COMMENT 'normalizer "NormalizerAuto"'
        -> ) Engine=Mroonga DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
    Query OK, 0 rows affected (0.19 sec)
    mysql> INSERT INTO diaries VALUES ("2013-04-23", "ブラックコーヒーを飲んだ。");
    Query OK, 1 row affected (0.00 sec)
    mysql> SELECT * FROM diaries
        ->        WHERE MATCH (content) AGAINST ("+ふらつく" IN BOOLEAN MODE);
    Empty set (0.00 sec)
    mysql> SELECT * FROM diaries
        ->        WHERE MATCH (content) AGAINST ("+ﾌﾞﾗｯｸ" IN BOOLEAN MODE);
    +------------+-----------------------------------------+
    | day        | content                                 |
    +------------+-----------------------------------------+
    | 2013-04-23 | ブラックコーヒーを飲んだ。 |
    +------------+-----------------------------------------+
    1 row in set (0.00 sec)

  How to get snippet (Keyword in context)

Mroonga provides functionality to get keyword in context. It is implemented as mroonga_snippet() <> UDF.

  Logging

Mroonga outputs the logs by default.

Log files are located in MySQL's data directory with the filename groonga.log.

Here is the example of the log.

    2010-10-07 17:32:39.209379|n|b1858f80|mroonga 1.10 started.
    2010-10-07 17:32:44.934048|d|46953940|hash get not found (key=test)
    2010-10-07 17:32:44.936113|d|46953940|hash put (key=test)

The default log level is NOTICE, i.e. we have important information only and we don't have debug information etc.).

You can get the log level by mroonga_log_level system variable, that is a global variable. You can also modify it dynamically by using SET phrase.

    mysql> SHOW VARIABLES LIKE 'mroonga_log_level';
    +-------------------+--------+
    | Variable_name     | Value  |
    +-------------------+--------+
    | mroonga_log_level | NOTICE |
    +-------------------+--------+
    1 row in set (0.00 sec)
    mysql> SET GLOBAL mroonga_log_level=DUMP;
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_log_level';
    +-------------------+-------+
    | Variable_name     | Value |
    +-------------------+-------+
    | mroonga_log_level | DUMP  |
    +-------------------+-------+
    1 row in set (0.00 sec)

Available log levels are the followings.

•   NONE

•   EMERG

•   ALERT

•   CRIT

•   ERROR

•   WARNING

•   NOTICE

•   INFO

•   DEBUG

•   DUMP

    See mroonga_log_level <#server-variable-mroonga-log-level> about details.

    You can reopen the log file by FLUSH LOGS. If you want to rotate the log file without stopping MySQL server, you can do in the following procedure.

1.  change the file name of groonga.log (by using OS's mv command etc.).

2.  invoke "FLUSH LOGS" in MySQL server (by mysql command or mysqladmin command).

  Next step

Now, you can use Mroonga as wrapper mode! If you want Mroonga to be faster, see also Optimizations <>.

FOOTNOTES

[1] MySQL 5.1 Reference Manual :: 11 Functions and Operations :: 11.7 Full-Text Search Functions <http://dev.mysql.com/doc/refman/5.1/ja/fulltext-search.html>

[2] In Groonga, we call it a 'tokenizer'.

REFERENCE

  Optimizations

Mroonga implemented some optimizations to return response faster.

Some optimizations can work only on Storage mode <>.

  Fetching only needed columns

This optimization can work only on Storage mode <>.

Groonga uses column store architecture. It means that Groonga doesn't need to fetch values of all columns for fetching a column value in a row. Groonga can fetch values of only needed columns.

InnoDB and MyISAM use row store architecture. They need to fetch values of all columns for fetching a column value in a row.

If you specify only columns you needed in SELECT, Mroonga just fetches only values of these columns. Mroonga doesn't fetch values of other columns.

Mroonga can work faster by reducing operations and I/O.

It's this optimization.

Here is a sample table definition to describe this optimization:

    CREATE TABLE t1 (
      c1 INT PRIMARY KEY AUTO_INCREMENT,
      c2 INT,
      c3 INT,
      ...
      c11 VARCHAR(20),
      c12 VARCHAR(20),
      ...
      c20 DATETIME
    ) ENGINE=Mroonga DEFAULT CHARSET=utf8;

Here is a SELECT to describe this optimization:

    SELECT c1, c2, c11 FROM t1 WHERE c2 = XX AND c12 = "XXX";

In this case, Mroonga fetches values from only c1, c2, c11 and c12. Mroonga doesn't fetch values from c3, c4, ..., c10, c13, ..., c19 and c20.

  Row count

This optimization can work only on Storage mode <>.

MySQL requires all column values from storage engine for processing COUNT(*) even if COUNT(*) doesn't need them.

Mroonga doesn't fetch any column values for the case.

Mroonga can work faster by reducing operations and I/O.

It's this optimization.

Here is a SELECT to describe this optimization:

    SELECT COUNT(*) FROM t1 WHERE MATCH(c2) AGAINST("+keyword" IN BOOLEAN MODE);

The SELECT fetches only COUNT(*) and condition in WHERE can be processed only by index. In this case, Mroonga uses this optimization.

You can confirm whether this optimization is used or not by looking Mroonga_count_skip <#status-variable-mroonga-count-skip> status variable:

    mysql> SHOW STATUS LIKE 'Mroonga_count_skip';
    +--------------------+-------+
    | Variable_name      | Value |
    +--------------------+-------+
    | Mroonga_count_skip | 1     |
    +--------------------+-------+
    1 row in set (0.00 sec)

Mroonga_count_skip <#status-variable-mroonga-count-skip> status variable is incremented when Mroonga uses this optimization.

You can disable this optimization by setting mroonga_enable_optimization <#server-variable-mroonga-enable-optimization> to false.

  ORDER BY LIMIT

This optimization can work on both Storage mode <> and Wrapper mode <>.

MySQL can process ORDER BY and LIMIT with low cost if you can get sorted records by index even if the number of matched records is very big.

MySQL can do the process for MATCH() AGAINST(IN NATURAL LANGUAGE MODE). But MySQL can't do the process for MATCH() AGAINST(IN BOOLEAN MODE).

It means that MySQL might take long time for MATCH() AGAINST(IN BOOLEAN MODE) that matches with many records.

Mroonga processes ORDER BY and LIMIT by Groonga and returns only target records to MySQL. It's very faster for query that matches with many records.

It's this optimization.

Here is a SELECT to describe this optimization:

    SELECT *
      FROM t1
     WHERE MATCH(c2) AGAINST("+keyword" IN BOOLEAN MODE)
     ORDER BY c1 LIMIT 1;

The SELECT runs full text search and sorts by Groonga and returns only one record to MySQL.

You can confirm whether this optimization is used or not by looking Mroonga_fast_order_limit <#status-variable-mroonga-fast-order-limit> status variable:

    mysql> SHOW STATUS LIKE 'Mroonga_fast_order_limit';
    +--------------------------+-------+
    | Variable_name            | Value |
    +--------------------------+-------+
    | Mroonga_fast_order_limit | 1     |
    +--------------------------+-------+
    1 row in set (0.00 sec)

Mroonga_fast_order_limit <#status-variable-mroonga-fast-order-limit> status variable is incremented when Mroonga uses this optimization.

This optimization is used only when all the following conditions are true:

•   Storage mode <>: WHERE clause has one MATCH AGAINST and zero or more arithmetic operations such as column < 100.

•   Wrapper mode <>: WHERE clause has only MATCH AGAINST.

•   No JOIN

•   No GROUP BY

•   No SQL_CALC_FOUND_ROWS

•   With LIMIT

•   Storage mode <>: ORDER BY clause has only columns or MATCH AGAINST that is used in WHERE clause.

•   Wrapper mode <>: ORDER BY clause has only primary column or MATCH AGAINST that is used in WHERE clause.

  SQL commands

This section describes avaiable SQL commands and unavailable SQL commands.

  List of available SQL commands

•   ALTER TABLE table_name ADD COLUMN column_name TEXT

•   ALTER TABLE table_name ADD FULLTEXT INDEX index_name (column_name)

•   ALTER TABLE table_name ADD SPATIAL KEY index_name (geometry_column_name)

•   ALTER TABLE table_name ENGINE = mroonga

•   ALTER TABLE table_name RENAME new_table_name

•   COMMIT

•   CREATE FULLTEXT INDEX index_name ON table_name(column_name)

•   CREATE FULLTEXT INDEX index_name ON table_name(column_name, ...)

•   CREATE TABLE IF NOT EXISTS table_name (...)

•   CREATE TABLE \`table-name-with-hyphen\` (...)

•   CREATE TABLE table_name (... FULLTEXT INDEX index_name (column_name) COMMENT 'tokenizer "TokenMecab"')

•   CREATE TABLE table_name (... FULLTEXT INDEX index_name (column_name) COMMENT 'normalizer "NormalizerMySQLUnicodeCIExceptKanaCIKanaWithVoicedSoundMark"')

•   CREATE TABLE table_name (... FULLTEXT INDEX index_name (column_name) COMMENT 'tokenizer "TokenDelimit", normalizer "NormalizerMySQLUnicodeCIExceptKanaCIKanaWithVoicedSoundMark"')

•   CREATE TABLE table_name (... FULLTEXT INDEX index_name (column_name))

•   CREATE TABLE table_name (... FULLTEXT INDEX index_name (column_name, ...))

•   CREATE TABLE table_name (... INDEX (column_name))

•   CREATE TABLE table_name (... INDEX (column_name, ...))

•   CREATE TABLE table_name (... INDEX USING BTREE (column_name))

•   CREATE TABLE table_name (... PRIMARY KEY (column_name))

•   CREATE TABLE table_name (...) CHARSET ASCII

•   CREATE TABLE table_name (...) CHARSET BINARY

•   CREATE TABLE table_name (...) CHARSET CP932

•   CREATE TABLE table_name (...) CHARSET EUCJPMS

•   CREATE TABLE table_name (...) CHARSET KOI8R

•   CREATE TABLE table_name (...) CHARSET LATIN1

•   CREATE TABLE table_name (...) CHARSET SJIS

•   CREATE TABLE table_name (...) CHARSET UJIS

•   CREATE TABLE table_name (...) CHARSET UTF8

•   CREATE TABLE table_name (...) CHARSET UTF8MB4

•   CREATE TABLE table_name (...) ENGINE=mroonga COMMENT = 'ENGINE "InnoDB"'

•   CREATE TABLE table_name (...) ENGINE=mroonga

•   CREATE TABLE table_name (\`_id\` INT)

•   CREATE TABLE table_name (\`_id\` INT, KEY(_id) USING HASH)

•   CREATE TABLE table_name (column_name BIGINT UNSIGNED)

•   CREATE TABLE table_name (column_name BIGINT)

•   CREATE TABLE table_name (column_name BINARY(...))

•   CREATE TABLE table_name (column_name BIT)

•   CREATE TABLE table_name (column_name BLOB)

•   CREATE TABLE table_name (column_name CHAR(...))

•   CREATE TABLE table_name (column_name DATE)

•   CREATE TABLE table_name (column_name DATETIME)

•   CREATE TABLE table_name (column_name DECIMAL)

•   CREATE TABLE table_name (column_name DOUBLE)

•   CREATE TABLE table_name (column_name ENUM(...))

•   CREATE TABLE table_name (column_name FLOAT)

•   CREATE TABLE table_name (column_name GEOMETRY NOT NULL)

•   CREATE TABLE table_name (column_name INT UNSIGNED)

•   CREATE TABLE table_name (column_name INT)

•   CREATE TABLE table_name (column_name LONGBLOB)

•   CREATE TABLE table_name (column_name LONGTEXT)

•   CREATE TABLE table_name (column_name MEDIUMBLOB)

•   CREATE TABLE table_name (column_name MEDIUMINT UNSIGNED)

•   CREATE TABLE table_name (column_name MEDIUMINT)

•   CREATE TABLE table_name (column_name MEDIUMTEXT)

•   CREATE TABLE table_name (column_name SET(...))

•   CREATE TABLE table_name (column_name SMALLINT UNSIGNED)

•   CREATE TABLE table_name (column_name SMALLINT)

•   CREATE TABLE table_name (column_name TEXT)

•   CREATE TABLE table_name (column_name TIME)

•   CREATE TABLE table_name (column_name TIMESTAMP)

•   CREATE TABLE table_name (column_name TINYBLOB)

•   CREATE TABLE table_name (column_name TINYINT UNSIGNED)

•   CREATE TABLE table_name (column_name TINYINT)

•   CREATE TABLE table_name (column_name TINYTEXT)

•   CREATE TABLE table_name (column_name VARBINARY(...))

•   CREATE TABLE table_name (column_name VARCHAR(...))

•   CREATE TABLE table_name (column_name YEAR)

•   CREATE TABLE table_name (column_name INT AUTO_INCREMENT)

•   CREATE TABLE table_name (column_name INT PRIMARY KEY)

•   CREATE TABLE table_name (column_name DATE PRIMARY KEY)

•   CREATE TABLE table_name (column_name DATETIME PRIMARY KEY)

•   CREATE TABLE table_name (column_name DATETIME(fractional_seconds_precision) PRIMARY KEY)

•   CREATE TABLE table_name (column_name DECIMAL PRIMARY KEY)

•   CREATE TABLE table_name (column_name DECIMAL(maximum_number_digits, fractional_seconds_precision) PRIMARY KEY)

•   CREATE TABLE table_name (column_name TIME PRIMARY KEY)

•   CREATE TABLE table_name (column_name TIME(fractional_seconds_precision) PRIMARY KEY)

•   CREATE TABLE table_name (column_name TIMESTAMP PRIMARY KEY)

•   CREATE TABLE table_name (column_name TIMESTAMP(fractional_seconds_precision) PRIMARY KEY)

•   CREATE TABLE table_name (column_name YEAR PRIMARY KEY)

•   CREATE TABLE table_name (column_name1 INT, column_name2 ..., KEY (column_name1, column_name2))

•   CREATE TABLE table_name (column_name1 DOUBLE, column_name2 ..., KEY (column_name1, column_name2))

•   CREATE TABLE table_name (column_name1 FLOAT, column_name2 ..., KEY (column_name1, column_name2))

•   CREATE TABLE table_name (column_name1 CHAR(...), column_name2 ..., KEY (column_name1, column_name2))

•   CREATE TABLE table_name (column_name1 VARCHAR(...), column_name2 ..., KEY (column_name1, column_name2))

•   CREATE TEMPORARY TABLE table_name (...)

•   DELETE FROM table_name WHERE ...

•   DROP INDEX column_name ON table_name

•   DROP TABLE IF EXISTS table_name, ...

•   DROP TABLE \`table-name-with-hyphen\`, ...

•   DROP TABLE table_name, ...

•   DROP TEMPORARY TABLE table_name

•   FLUSH LOGS

•   FLUSH TABLES

•   INSERT INTO (geometry_column_name) VALUES (GeomFromText('POINT(...)'))

•   INSERT INTO ... VALUES ...

•   INSERT INTO ... VALUES ... ON DUPLICATE KEY UPDATE ... (for PRIMARY KEY)

•   INSERT INTO ... VALUES ... ON DUPLICATE KEY UPDATE ... (for UNIQUE KEY)

•   INSERT INTO table_name (column_name, ...) SELECT ... FROM other_table_name

•   REPLACE INTO table_name SELECT ... FROM other_table_name ...

•   SELECT * FROM information_schema.plugins WHERE plugin_name = "mroonga"

•   SELECT * FROM table_name

•   SELECT * FROM table_name FORCE INDEX(index_name) WHERE ...

•   SELECT * FROM table_name ORDER BY column_name ASC LIMIT ...

•   SELECT * FROM table_name ORDER BY column_name ASC

•   SELECT * FROM table_name ORDER BY column_name DESC LIMIT ...

•   SELECT * FROM table_name ORDER BY column_name DESC

•   SELECT * FROM table_name ORDER BY column_name

•   SELECT * FROM table_name WHERE MATCH(column_name) AGAINST("*D+ word ..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MATCH(column_name) AGAINST("*D- word ..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MATCH(column_name) AGAINST("*DOR word ..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MATCH(column_name) AGAINST("*W word ..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MATCH(column_name) AGAINST("+word ..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MATCH(column_name) AGAINST("..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MATCH(column_name, ...) AGAINST("..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE MBRContains(GeomFromText('LineString(...)'), geometry_column_name)

•   SELECT * FROM table_name WHERE NOT MATCH(column_name) AGAINST("..." IN BOOLEAN MODE)

•   SELECT * FROM table_name WHERE column_name < ...

•   SELECT * FROM table_name WHERE column_name <= ...

•   SELECT * FROM table_name WHERE column_name = ...

•   SELECT * FROM table_name WHERE column_name > ...

•   SELECT * FROM table_name WHERE column_name >= ...

•   SELECT * FROM table_name WHERE column_name BETWEEN ... AND ...

•   SELECT * FROM table_name WHERE column_name IN (SELECT sub_column_name FROM sub_table_name WHERE MATCH(sub_text_column_name) AGAINST("..."))

•   SELECT *, MATCH(column_name) AGAINST("..." IN BOOLEAN MODE) FROM table_name WHERE MATCH(column_name) AGAINST("..." IN BOOLEAN MODE)

•   SELECT FOUND_ROWS()

•   SELECT SQL_CALC_FOUND_ROWS * FROM table_name WHERE MATCH(...) AGAINST("..." IN BOOLEAN MODE) ORDER BY column_name LIMIT start,n_records

•   SELECT last_insert_grn_id()

•   SET binlog_format="MIXED"

•   SET binlog_format="ROW"

•   SET binlog_format="STATEMENT"

•   TRUNCATE TABLE table_name

•   UPDATE table_name SET column_name = ...

  List of unavailable SQL commands

•   CREATE TABLE (...) CHARSET not_listed_charset_above

•   INSERT INTO (geometry_column_name) VALUES (GeomFromText('LineString(...)'))

•   INSERT INTO (...) VALUES (null)

•   START TRANSACTION

    Basically, The character set which Groonga supports (EUC-JP/UTF-8/SJIS/LATIN1/KOI8R) is also supported in Mroonga.

  I found an unlisted SQL command

If you find unlisted SQL command and confirm whether the SQL command works well or not, we want to list it. Please mail it to us <https://groonga.org/docs/community.html> or send a patch against the source file <https://github.com/mroonga/mroonga/blob/main/doc/source/reference.rst> by pull request system on GitHub.

  Server variables

Here are the explanations of server variables that are introduced by Mroonga.

  mroonga_action_on_fulltext_query_error

The default behavior of fulltext query error.

The default value of mroonga_action_on_fulltext_query_error is ERROR_AND_LOG. This is the conventional behavior which is equal to old version of mroonga.

Here is the list of mroonga_action_on_fulltext_query_error which you can use.
Value | Description
ERROR | Report an error. Logging is disabled.
ERROR_AND_LOG | Report an error. Logging is enabled. (This is the default)
IGNORE | Just ignore an error. Logging is disabled.
IGNORE_AND_LOG | Ignore an error, but logging is enabled. (Similar to InnoDB behavior)

Here is an example SQL to confirm the value of mroonga_action_on_fulltext_query_error:

    mysql> SHOW VARIABLES LIKE 'mroonga_action_on_fulltext_query_error';
    +----------------------------------------+---------------+
    | Variable_name                          | Value         |
    +----------------------------------------+---------------+
    | mroonga_action_on_fulltext_query_error | ERROR_AND_LOG |
    +----------------------------------------+---------------+
    1 row in set (0.00 sec)

  mroonga_boolean_mode_syntax_flags

The flags to custom syntax in MATCH () AGAINST ('...' IN BOOLEAN MODE).

This variable is system and session variable.

Here are available flags:
Flag | Description
DEFAULT | Equals to SYNTAX_QUERY,ALLOW_LEADING_NOT.
SYNTAX_QUERY | Uses query syntax in Groonga <https://groonga.org/docs/reference/grn_expr/query_syntax.html>. Query syntax in Groonga is a compatible syntax with MySQL's BOOLEAM MODE syntax. If neither SYNTAX_QUERY nor SYNTAX_SCRIPT aren't specified, SYNTAX_QUERY is used.
SYNTAX_SCRIPT | Uses script syntax in Groonga <https://groonga.org/docs/reference/grn_expr/script_syntax.html>. It's JavaScript like syntax. You can use full Groonga features with this syntax. If both SYNTAX_QUERY and SYNTAX_SCRIPT are specified, SYNTAX_SCRIPT is used.
ALLOW_COLUMN | Allows COLUMN:... syntax in query syntax. It's not compatible with MySQL's BOOLEAM MODE syntax. You can use multiple indexes in one MATCH () AGAINST () with this syntax. MySQL can use only one index in a query. You can avoid the restriction by this syntax. You can use not only full-text search operation but also other more operations such as equal operation and prefix search operation with this syntax. See query syntax in Groonga <https://groonga.org/docs/reference/grn_expr/query_syntax.html> for details.
ALLOW_UPDATE | Allows updating value by COLUMN:=NEW_VALUE syntax in query syntax.
ALLOW_LEADING_NOT | Allows -NOT_INCLUDED_KEYWORD ... syntax in query syntax.

The default flags is DEFAULT. It is MySQL's BOOLEAN MODE compatible syntax.

You can combine flags by separated by comma such as SYNTAX_QUERY,ALLOW_LEADING_NOT.

Here is an example SQL to use script syntax in Groonga:

    mysql> SET mroonga_boolean_mode_syntax_flags = "SYNTAX_SCRIPT";

  mroonga_database_path_prefix

TODO:

  mroonga_default_parser

Deprecated since version 5.04: Use mroonga_default_tokenizer instead.

The default parser of the full text search. The default value can be specified by --with-default-parser=PARSER configure argument, whose default value is TokenBigram.

Here is an example to use TokenBigramSplitSymbolAlphaDigit as a fulltext search parser. It is used by body_index fulltext index.

    SET GLOBAL mroonga_default_parser=TokenBigramSplitSymbolAlphaDigit;
    CREATE TABLE diaries (
      id INT PRIMARY KEY AUTO_INCREMENT,
      body TEXT,
      FULLTEXT INDEX body_index (body)
    ) DEFAULT CHARSET UTF8;

  mroonga_default_tokenizer

Added in version 5.04.

The default tokenizer of the full text search. The default value can be specified by --with-default-tokenizer=TOKENIZER configure argument, whose default value is TokenBigram.

Here is an example to use TokenBigramSplitSymbolAlphaDigit as a fulltext index tokenizer. It is used by body_index fulltext index.

    SET GLOBAL mroonga_default_tokenizer=TokenBigramSplitSymbolAlphaDigit;
    CREATE TABLE diaries (
      id INT PRIMARY KEY AUTO_INCREMENT,
      body TEXT,
      FULLTEXT INDEX body_index (body)
    ) DEFAULT CHARSET UTF8;

  mroonga_default_wrapper_engine

TODO:

  mroonga_dry_write

Whether really write data to Groonga database or not. The default value is OFF that means data are really written to Groonga database. Usually we don't need to change the value of this variable. This variable is useful for benchmark because we can measure processing time MySQL and Mroonga. It doesn't include Groonga's processing time.

Here is an example SQL to disable writing data to Groonga database:

    mysql> SHOW VARIABLES LIKE 'mroonga_dry_write';
    +-------------------+-------+
    | Variable_name     | Value |
    +-------------------+-------+
    | mroonga_dry_write | OFF   |
    +-------------------+-------+
    1 row in set (0.00 sec)
    mysql> SET mroonga_dry_write = true;
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_dry_write';
    +-------------------+-------+
    | Variable_name     | Value |
    +-------------------+-------+
    | mroonga_dry_write | ON    |
    +-------------------+-------+
    1 row in set (0.00 sec)

  mroonga_enable_optimization

Whether enable optimization or not. The default value is ON that means optimization is enabled. Usually we don't need to change the value of this variable. This variable is useful for benchmark.

Here is an example SQL to disable optimization:

    mysql> SHOW VARIABLES LIKE 'mroonga_enable_optimization';
    +-----------------------------+-------+
    | Variable_name               | Value |
    +-----------------------------+-------+
    | mroonga_enable_optimization | ON    |
    +-----------------------------+-------+
    1 row in set (0.00 sec)
    mysql> SET mroonga_enable_optimization = false;
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_enable_optimization';
    +-----------------------------+-------+
    | Variable_name               | Value |
    +-----------------------------+-------+
    | mroonga_enable_optimization | OFF   |
    +-----------------------------+-------+
    1 row in set (0.00 sec)

  mroonga_libgroonga_support_lz4

The status of libgroonga supports LZ4.

Here is an example SQL to confirm the status of libgroonga supports LZ4:

    mysql> SHOW GLOBAL VARIABLES LIKE 'mroonga_libgroonga_support_lz4';
    +--------------------------------+-------+
    | Variable_name                  | Value |
    +--------------------------------+-------+
    | mroonga_libgroonga_support_lz4 | ON    |
    +--------------------------------+-------+

  mroonga_libgroonga_support_zlib

The status of libgroonga supports zlib.

Here is an example SQL to confirm the status of libgroonga supports zlib:

    mysql> SHOW GLOBAL VARIABLES LIKE 'mroonga_libgroonga_support_zlib';
    +---------------------------------+-------+
    | Variable_name                   | Value |
    +---------------------------------+-------+
    | mroonga_libgroonga_support_zlib | ON    |
    +---------------------------------+-------+

  mroonga_libgroonga_support_zstd

The status of libgroonga supports Zstandard.

Here is an example SQL to confirm the status of libgroonga supports Zstandard:

    mysql> SHOW GLOBAL VARIABLES LIKE 'mroonga_libgroonga_support_zstd';
    +---------------------------------+-------+
    | Variable_name                   | Value |
    +---------------------------------+-------+
    | mroonga_libgroonga_support_zstd | ON    |
    +---------------------------------+-------+

  mroonga_libgroonga_support_mecab

The status of libgroonga supports MeCab.

Here is an example SQL to confirm the status of libgroonga supports MeCab.

    mysql> SHOW GLOBAL VARIABLES LIKE 'mroonga_libgroonga_support_mecab';
    +----------------------------------+-------+
    | Variable_name                    | Value |
    +----------------------------------+-------+
    | mroonga_libgroonga_support_mecab | ON    |
    +----------------------------------+-------+

  mroonga_libgroonga_version

The version string of the groonga library.

Here is an example SQL to confirm the using groonga library version:

    mysql> SHOW VARIABLES LIKE 'mroonga_libgroonga_version';
    +----------------------------+------------------+
    | Variable_name              | Value            |
    +----------------------------+------------------+
    | mroonga_libgroonga_version | 1.2.8-9-gbf05b82 |
    +----------------------------+------------------+
    1 row in set (0.00 sec)

  mroonga_lock_timeout

TODO:

  mroonga_log_file

The path of the log file of Mroonga. The default value is groonga.log.

Here is an example transcript to change log file to /tmp/mroonga.log:

    mysql> SHOW VARIABLES LIKE 'mroonga_log_file';
    +------------------+-------------+
    | Variable_name    | Value       |
    +------------------+-------------+
    | mroonga_log_file | groonga.log |
    +------------------+-------------+
    1 row in set (0.00 sec)
    mysql> SET GLOBAL mroonga_log_file = "/tmp/mroonga.log";
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_log_file';
    +------------------+------------------+
    | Variable_name    | Value            |
    +------------------+------------------+
    | mroonga_log_file | /tmp/mroonga.log |
    +------------------+------------------+
    1 row in set (0.00 sec)

  mroonga_log_level

The output level of Mroonga log file. The default value is NOTICE.

Here is the list of mroonga_log_level which you can use.
Log level | Description
NONE | No logging output.
EMERG | Logging emergency messages such as database corruption.
ALERT | Logging alert messages such as internal error.
CRIT | Logging critical message such as deadlock.
ERROR | Logging error messages such as API error which Mroonga use.
WARNING | Logging warning messages such as invalid argument.
NOTICE | Logging notice messages such as configuration or status changed.
INFO | Logging informative messages such as file system operation.
DEBUG | Logging debug messages. Recommend to use for Mroonga developer or bug reporter.
DUMP | Logging dump messages.

Here is an example transcript to change log level to DEBUG that logs many messages for debugging:

    mysql> SHOW VARIABLES LIKE 'mroonga_log_level';
    +-------------------+--------+
    | Variable_name     | Value  |
    +-------------------+--------+
    | mroonga_log_level | NOTICE |
    +-------------------+--------+
    1 row in set (0.00 sec)
    mysql> SET GLOBAL mroonga_log_level = "debug";
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_log_level';
    +-------------------+-------+
    | Variable_name     | Value |
    +-------------------+-------+
    | mroonga_log_level | DEBUG |
    +-------------------+-------+
    1 row in set (0.00 sec)

  mroonga_query_log_file

The path of the query log file of Mroonga. The default value is empty.

If this value is empty, the query log is not stored to file. If this value is not empty, query log is stored to the specified file.

Here is an example transcript to change query log file to /tmp/mroonga_query.log:

    mysql> SHOW VARIABLES LIKE 'mroonga_query_log_file';
    +------------------------+-------+
    | Variable_name          | Value |
    +------------------------+-------+
    | mroonga_query_log_file |       |
    +------------------------+-------+
    1 row in set (0.00 sec)
    mysql> SET GLOBAL mroonga_query_log_file = "/tmp/mroonga.log";
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_query_log_file';
    +------------------------+------------------+
    | Variable_name          | Value            |
    +------------------------+------------------+
    | mroonga_query_log_file | /tmp/mroonga.log |
    +------------------------+------------------+
    1 row in set (0.00 sec)

  mroonga_n_workers

Added in version 15.00.

Specify the number of Groonga's internal processing in parallel.

In Mroonga, when parallel processing is enabled, CREATE INDEX is processed in parallel, which is expected to speed up the process.

Here is the list of values which you can use. The default value is 0.
Value | Description
0 or 1 | Execute in serial.
2 or more | Execute in parallel with at most the specified number of threads.
-1 | Execute in parallel with the threads of at most the number of CPU cores.

Here is an example SQL to change the number of parallels to 2:

    mysql> SHOW VARIABLES LIKE 'mroonga_n_workers';
    +-------------------+-------+
    | Variable_name     | Value |
    +-------------------+-------+
    | mroonga_n_workers | 0     |
    +-------------------+-------+
    1 row in set (0.001 sec)
    mysql> SET GLOBAL mroonga_n_workers = 2;
    Query OK, 0 rows affected (0.000 sec)
    mysql> SHOW VARIABLES LIKE 'mroonga_n_workers';
    +-------------------+-------+
    | Variable_name     | Value |
    +-------------------+-------+
    | mroonga_n_workers | 2     |
    +-------------------+-------+
    1 row in set (0.001 sec)

Only available in GLOBAL.

If set with SET GLOBAL, it will enable from the next session. Note that this is not enabled for the current session. If you set it persistently, we recommend that you use my.cnf to set it.

  mroonga_match_escalation_threshold

The threshold to determin whether match method is escalated. See search specification for Groonga <https://groonga.org/docs/spec/search.html> about match method escalation.

The default value is the same as Groonga's default value. It's 0 for the default installation. The dafault value can be configured in my.cnf or by SET GLOBAL mroonga_match_escalation_threshold = THRESHOLD;. Because this variable's scope is both global and session.

Here is an example to use -1 as a threshold to determin whether match method is escalated. -1 means that never escalated.

    SET GLOBAL mroonga_match_escalation_threshold = -1;

Here is an another example to show behavior change by the variable value.

    CREATE TABLE diaries (
      id INT PRIMARY KEY AUTO_INCREMENT,
      title TEXT,
      tags TEXT,
      FULLTEXT INDEX tags_index (tags) COMMENT 'tokenizer "TokenDelimit"'
    ) ENGINE=mroonga DEFAULT CHARSET=UTF8;
    -- Test data
    INSERT INTO diaries (title, tags) VALUES ("Hello groonga!", "groonga install");
    INSERT INTO diaries (title, tags) VALUES ("Hello mroonga!", "mroonga install");
    -- Matches all records that have "install" tag.
    SELECT * FROM diaries WHERE MATCH (tags) AGAINST ("install" IN BOOLEAN MODE);
    -- id        title   tags
    -- 1 Hello groonga!  groonga install
    -- 2 Hello mroonga!  mroonga install
    -- Matches no records by "gr" tag search because no "gr" tag is used.
    -- But matches a record that has "groonga" tag because search
    -- method is escalated and prefix search with "gr" is used.
    -- The default threshold is 0. It means that no records are matched then
    -- search method is escalated.
    SELECT * FROM diaries WHERE MATCH (tags) AGAINST ("gr" IN BOOLEAN MODE);
    -- id        title   tags
    -- 1 Hello groonga!  groonga install
    -- Disables escalation.
    SET mroonga_match_escalation_threshold = -1;
    -- No records are matched.
    SELECT * FROM diaries WHERE MATCH (tags) AGAINST ("gr" IN BOOLEAN MODE);
    -- id        title   tags
    -- Enables escalation again.
    SET mroonga_match_escalation_threshold = 0;
    -- Matches a record by prefix search with "gr".
    SELECT * FROM diaries WHERE MATCH (tags) AGAINST ("gr" IN BOOLEAN MODE);
    -- id        title   tags
    -- 1 Hello groonga!  groonga install

  mroonga_max_n_records_for_estimate

Added in version 5.02.

TODO:

  mroonga_enable_operations_recording

Whether recording operations for recover is enabled or not. The default value is OFF that means operations are not recorded to Groonga database. It needs to reopen the database with FLUSH TABLES in order to reflect the variable is changed.

Here is an example SQL to disable operations recording:

    mysql> SET GLOBAL mroonga_enable_operations_recording = false;
    Query OK, 0 rows affected (0.00 sec)
    mysql> FLUSH TABLES;
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW GLOBAL VARIABLES LIKE 'mroonga_enable_operations_recording';
    +-------------------------------------+-------+
    | Variable_name                       | Value |
    +-------------------------------------+-------+
    | mroonga_enable_operations_recording | OFF   |
    +-------------------------------------+-------+

  mroonga_vector_column_delimiter

The delimiter when outputting a vector column. The default value is a white space.

Here is an example SQL to change the delimiter to a semicolon from a white space:

    mysql> SHOW VARIABLES LIKE 'mroonga_vector_column_delimiter';
    +---------------------------------+-------+
    | Variable_name                   | Value |
    +---------------------------------+-------+
    | mroonga_vector_column_delimiter |       |
    +---------------------------------+-------+
    1 row in set (0.00 sec)
    mysql> SET GLOBAL mroonga_vector_column_delimiter = ';';
    Query OK, 0 rows affected (0.00 sec)
    mysql> SHOW GLOBAL VARIABLES LIKE 'mroonga_vector_column_delimiter';
    +---------------------------------+-------+
    | Variable_name                   | Value |
    +---------------------------------+-------+
    | mroonga_vector_column_delimiter | ;     |
    +---------------------------------+-------+

  mroonga_version

The version string of Mroonga.

Here is an example SQL to confirm the running mroonga version:

    mysql> SHOW VARIABLES LIKE 'mroonga_version';
    +-----------------+-------+
    | Variable_name   | Value |
    +-----------------+-------+
    | mroonga_version | 1.10  |
    +-----------------+-------+
    1 row in set (0.00 sec)

  mroonga_condition_push_down_type

Added in version 7.10.

It controls how to enable condition push down support.

The default value is ONE_FULL_TEXT_SEARCH. It means that condition push down is enabled only when WHERE clause has one MATCH AGAINST condition.

Here are available values:
Value | Description
NONE | Never use condition push down.
ALL | Always use condition push down. It's experimental for now.
ONE_FULL_TEXT_SEARCH | Use condition push down only when one MATCH AGAINST condition exists in WHERE. It's the default.

Here is an example SQL to confirm the current value:

    mysql> SHOW VARIABLES LIKE 'mroonga_condition_push_down_type';
    +----------------------------------+----------------------+
    | Variable_name                    | Value                |
    +----------------------------------+----------------------+
    | mroonga_condition_push_down_type | ONE_FULL_TEXT_SEARCH |
    +----------------------------------+----------------------+
    1 row in set (0.00 sec)

  Status variables

Here are the explanations of status variables that are introduced by Mroonga.

  Mroonga_count_skip

This value is increased when 'fast line count feature' is used. You can use this value to check if the feature is working when you enable it.

Here is an example how to check it:

    mysql> SHOW STATUS LIKE 'Mroonga_count_skip';
    +--------------------+-------+
    | Variable_name      | Value |
    +--------------------+-------+
    | Mroonga_count_skip | 0     |
    +--------------------+-------+
    1 row in set (0.00 sec)

  Mroonga_fast_order_limit

This value is increased when 'fast ORDER BY LIMIT feature' is used. You can use this value to check if the feature is working when you enable it.

Here is an example how to check it:

    mysql> SHOW STATUS LIKE 'Mroonga_fast_order_limit';
    +--------------------------+-------+
    | Variable_name            | Value |
    +--------------------------+-------+
    | Mroonga_fast_order_limit | 0     |
    +--------------------------+-------+
    1 row in set (0.00 sec)

  Mroonga_condition_push_down

This value is increased when condition push down is used. You can use this value to check whether condition push down is used or not.

Here is an example how to check it:

    mysql> SHOW STATUS LIKE 'Mroonga_condition_push_down';
    +-----------------------------+-------+
    | Variable_name               | Value |
    +-----------------------------+-------+
    | Mroonga_condition_push_down | 1     |
    +-----------------------------+-------+
    1 row in set (0.00 sec)

  Mroonga_n_pooling_contexts

This value shows the number of pooling contexts for mroonga_command() <>. These contexts are reused from multiple mroonga_command() <> calls. So, it's not increased continually.

Here is an example how to check it:

    mysql> SHOW STATUS LIKE 'Mroonga_n_pooling_contexts';
    +----------------------------+-------+
    | Variable_name              | Value |
    +----------------------------+-------+
    | Mroonga_n_pooling_contexts | 0     |
    +----------------------------+-------+
    1 row in set (0.00 sec)

  Mroonga_memory_map_size

Added in version 12.0.4.

This value shows the currently memory mapped size in bytes.

Here is an example how to check it:

    mysql> SHOW STATUS LIKE 'Mroonga_memory_map_size';
    +-------------------------+----------+
    | Variable_name           | Value    |
    +-------------------------+----------+
    | Mroonga_memory_map_size | 83406848 |
    +-------------------------+----------+
    1 row in set (0.00 sec)

  UDF (User Defined Functions)

Mroonga provides some useful functionality as UDF (User Defined Functions).

  mroonga_command()

Added in version 3.02.

  Summary

mroonga_command() UDF executes the given string as a Groonga command and returns result of the Groonga command. Groonga command will be faster than MySQL query.

mroonga_command() is an UDF for advanced users. Normally, you don't need to use this UDF.

  Syntax

mroonga_command() has only one required parameter:

    mroonga_command(command)

command is a string value. It's a Groonga command to be executed.

  Usage

TODO

  Parameters

  Required parameters

There is one required parameter, command.

  command

It specifies a Groonga command to be executed.

  Return value

It returns an evaluated result of the given Groonga command as a string.

  See also

Command <https://groonga.org/docs/reference/command.html> in Groonga document.

  mroonga_escape()

Added in version 3.08.

  Summary

mroonga_escape UDF provides functionality to escape given string. It also accepts parameter what character should be escaped.

  Syntax

mroonga_escape() has required parameter and optional parameter:

    mroonga_escape(string)
    mroonga_escape(string, special_characters)

  Usage

Here is the example query which use special characters to be escaped:

    SELECT * FROM `symbols` WHERE MATCH(`content`) AGAINST(mroonga_escape("+hello_world()", "()") IN BOOLEAN MODE);

Here is the example about special characters which is escaped:

    SELECT mroonga_escape("+-<>~*()\"\\\:");
    -- \+\-\<\>\~\*\(\)\"\\\:

  Parameters

  Required parameters

There is one required parameter, string.

  string

It specifies text which you want to escape.

  Optional parameters

There is one optional parameter, special_characters.

  special_characters

It specifies characters to escape.

The default value is +-<>~*()"\\:.

  Return value

It returns escaped string.

  mroonga_highlight_html()

Added in version 7.05.

  Summary

mroonga_highlight_html() highlights the specified keywords in target text. It surrounds each keyword with <span class="keyword">...</span> and special characters in HTML such as < and > are escaped. You can use the result as is safely in HTML.

  Syntax

mroonga_highlight_html() has required parameter and optional parameter:

    mroonga_highlight_html(text, query AS query)
    mroonga_highlight_html(text, query AS query, open_tag AS open_tag)
    mroonga_highlight_html(text, query AS query, close_tag AS close_tag)
    mroonga_highlight_html(text, query AS query, open_tag AS open_tag, close_tag AS close_tag)
    mroonga_highlight_html(text, keyword1, ..., keywordN)
    mroonga_highlight_html(text, keyword1, ..., keywordN, open_tag AS open_tag)
    mroonga_highlight_html(text, keyword1, ..., keywordN, open_tag AS close_tag)
    mroonga_highlight_html(text, keyword1, ..., keywordN, open_tag AS open_tag, close_tag AS close_tag)

AS query is very important. You must specify it to extract keywords from query.

open_tag and close_tag are optional. You can specify a tag for highlighting.

  Usage

Here is a sample to highlight keywords "mroonga" and "groonga" in target text by query "mroonga OR groonga". You must specify AS query:

    SELECT mroonga_highlight_html('Mroonga is the Groonga based storage engine.',
                                  'mroonga OR groonga' AS query) AS highlighted;

Here is the result of the execution example:

    +--------------------------------------------------------------------------------------------------------+
    | highlighted                                                                                            |
    +--------------------------------------------------------------------------------------------------------+
    | <span class="keyword">Mroonga</span> is the <span class="keyword">Groonga</span> based storage engine. |
    +--------------------------------------------------------------------------------------------------------+
    1 row in set (0.00 sec)

Here is a sample to highlight keywords "mroonga" and "groonga" in target text by keywords "mroonga" and "groonga":

    SELECT mroonga_highlight_html('Mroonga is the Groonga based storage engine.',
                                  'mroonga', 'groonga') AS highlighted;

Here is the result of the execution example:

    +--------------------------------------------------------------------------------------------------------+
    | highlighted                                                                                            |
    +--------------------------------------------------------------------------------------------------------+
    | <span class="keyword">Mroonga</span> is the <span class="keyword">Groonga</span> based storage engine. |
    +--------------------------------------------------------------------------------------------------------+

  Parameters

  Required parameters

There is one required parameter.

  text

The column name of string or string value to be highlighted.

  Optional parameters

There are some optional parameters.

  query

Specify query in Groonga's query syntax <https://groonga.org/docs/reference/grn_expr/query_syntax.html>.

You must specify AS query to extract keywords from query like the following:

    SELECT mroonga_highlight_html('...', 'mroonga OR groonga' AS query);

  keyword

Specify 0 or more keywords to be highlighted.

  open_tag

Specify an open tag for highlighting.

You must specify AS open_tag to specify this parameter like the following.

    SELECT mroonga_highlight_html('Mroonga is the Groonga based storage engine.', 'groonga',
                                  '<span class="my-class">' AS open_tag) AS highlighted;
    -- +----------------------------------------------------------------------------+
    -- | highlighted                                                                |
    -- +----------------------------------------------------------------------------+
    -- | Mroonga is the <span class="my-class">Groonga</span> based storage engine. |
    -- +----------------------------------------------------------------------------+

The default value is <span class="keyword">.

  close_tag

Specify a close tag for highlighting.

You must specify AS close_tag to specify this parameter like the following.

    SELECT mroonga_highlight_html('Mroonga is the Groonga based storage engine.', 'groonga',
                                  '<mark>' AS open_tag, '</mark>' AS close_tag) AS highlighted;
    -- +-----------------------------------------------------------+
    -- | highlighted                                               |
    -- +-----------------------------------------------------------+
    -- | Mroonga is the <mark>Groonga</mark> based storage engine. |
    -- +-----------------------------------------------------------+

The default value is </span>.

  Return value

It returns highlighted HTML. If optional parameter is not given, it only escapes special characters in HTML such as <, > in text.

  mroonga_last_insert_grn_id()

  Summary

mroonga_last_insert_grn_id() UDF provides functionality to get id of last inserted record.

This id is unique in Groonga.

  Syntax

  Usage

  Parameters

  Required parameters

  Return value

It returns id of last inserted record.

  mroonga_normalize()

Added in version 5.11.

  Summary

mroonga_normalize() UDF normalizes text by Groonga's normalizer.

  Syntax

mroonga_normalize() has required parameter and optional parameter:

    mroonga_normalize(string)
    mroonga_normalize(string, normalizer_name)

  Usage

Here is the example query which use Groonga's NormalizerAuto normalizer to be normalized:

    SELECT mroonga_normalize("ABCDあぃうぇ㍑");
    abcdあぃうぇリットル

Here is the example query which use Groonga's NormalizerMySQLUnicodeCIExceptKanaCIKanaWithVoicedSoundMark normalizer to be normalized:

    SELECT mroonga_normalize("aBｃＤあぃウェ㍑", "NormalizerMySQLUnicodeCIExceptKanaCIKanaWithVoicedSoundMark");
    ABCDあぃうぇ㍑

  Parameters

  Required parameters

There is one required parameter, string.

  string

It specifies text which you want to normalize.

  Optional parameters

There is one optional parameter, normalizer_name.

  normalizer_name

It specifies Groonga's normalizer name to normalize.

The default value is NormalizerAuto.

  Return value

It returns normalized string.

  mroonga_snippet()

Added in version 2.07.

  Summary

mroonga_snippet() UDF provides functionality to get keyword in context.

  Syntax

mroonga_snippet() has required parameter and optional parameter:

    mroonga_snippet(document,
                    max_length,
                    max_count,
                    encoding,
                    skip_leading_spaces,
                    html_escape,
                    snippet_prefix,
                    snippet_suffix,
                    word1, word1_prefix, word1_suffix,
                    ...,
                    [wordN, wordN_prefix, wordN_suffix])

  Usage

There is a case that you want to extract keyword and surrounding text as a search results.

Snippet means 'keyword and surrounding text'. It is called 'Keyword in context'.

mroonga_snippet() function provides the way to get snippet from search results.

You can use mroonga_snippet() UDF in storage mode and wrapper mode.

mroonga_snippet() function is included in Mroonga as a User-Defined Function (UDF), but if you have not yet register it in MySQL by CREATE FUNCTION, you need to invoke the following SQL for defining a function.

    mysql> CREATE FUNCTION mroonga_snippet RETURNS STRING SONAME 'ha_mroonga.so';

mroonga_snippet() function is useful for searching the text which contains keyword and associated one by using MATCH .. AGAINST syntax.

Imagine searching the document which contains 'fulltext' as a keyword. Assume that some keyword such as 'MySQL' and 'search' are associated with 'fulltext'.

mroonga_snippet() function meets above.

Here is the schema definition for execution examples(storage mode):

    CREATE TABLE `snippet_test` (
      `id` int(11) NOT NULL,
      `text` text,
      PRIMARY KEY (`id`),
      FULLTEXT KEY `text` (`text`)
    ) ENGINE=mroonga DEFAULT CHARSET=utf8

Here is the schema definition for execution examples(wrapper mode):

    CREATE TABLE `snippet_test` (
      `id` int(11) NOT NULL,
      `text` text,
      PRIMARY KEY (`id`),
      FULLTEXT KEY `text` (`text`)
    ) ENGINE=mroonga COMMENT 'engine = "innodb"' DEFAULT CHARSET=utf8

Here is the sample data for execution examples:

    INSERt INTO snippet_test (id, text) VALUES (1, 'An open-source fulltext search engine and column store.');
    INSERT INTO snippet_test (id, text) VALUES (2, 'An open-source storage engine for fast fulltext search with MySQL.');
    INSERT INTO snippet_test (id, text) VALUES (3, 'Tritonn is a patched version of MySQL that supports better fulltext search function with Senna.');

Here is the results of execution examples:

    mysql> SELECT * FROM snippet_test;
    +----+-------------------------------------------------------------------------------------------------+
    | id | text                                                                                            |
    +----+-------------------------------------------------------------------------------------------------+
    |  1 | An open-source fulltext search engine and column store.                                         |
    |  2 | An open-source storage engine for fast fulltext search with MySQL.                              |
    |  3 | Tritonn is a patched version of MySQL that supports better fulltext search function with Senna. |
    +----+-------------------------------------------------------------------------------------------------+
    3 rows in set (0.00 sec)
    mysql> SELECT id, text, mroonga_snippet(text, 8, 2, 'ascii_general_ci', 1, 1, '...', '...<br>', 'fulltext', '<span class="w1">', '</span>', 'MySQL', '<span class="w2">', '</span>', 'search', '<span calss="w3">', '</span>') FROM snippet_test WHERE MATCH(text) AGAINST ('+fulltext' IN BOOLEAN MODE);
    +----+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | id | text                                                                                            | mroonga_snippet(text, 8, 2, 'ascii_general_ci', 1, 1, '...', '...<br>', 'fulltext', '<span class="w1">', '</span>', 'MySQL', '<span class="w2">', '</span>', 'search', '<span calss="w3">', '</span>') |
    +----+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    |  1 | An open-source fulltext search engine and column store.                                         | ...<span class="w1">fulltext</span>...<br>... <span calss="w3">search</span> ...<br>                                                                                                                   |
    |  2 | An open-source storage engine for fast fulltext search with MySQL.                              | ...<span class="w1">fulltext</span>...<br>... <span calss="w3">search</span> ...<br>                                                                                                                   |
    |  3 | Tritonn is a patched version of MySQL that supports better fulltext search function with Senna. | ...f <span class="w2">MySQL</span> ...<br>...<span class="w1">fulltext</span>...<br>                                                                                                                   |
    +----+-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    3 rows in set (0.00 sec)

The keyword 'fulltext' and associated keyword 'MySQL' and 'search' has been extracted.

  Parameters

  Required parameters

There are many required parameters.

  document

The column name or string value is required.

  max_length

The max length of snippet (bytes) is required.

  max_count

The max elements of snippets (N word) is required.

  encoding

The encoding of document is required. You can specify the value of encoding such as 'ascii_general_ci', 'cp932_japanese_ci', 'eucjpms_japanese_ci' and so on.

  skip_leading_spaces

Specify whether skip leading spaces or not. Specify the value 1 for skipping leading spaces, 0 for not.

  html_escape

HTML escape is enabled or not. Specify the value 1 for enabling HTML escape, 0 for not.

  prefix

The start text of snippet.

  suffix

The end text of snippet.

  Optional parameters

There is one optional group parameter, wordN related prefix and suffix.

  wordN

Specify any word.

  wordN_prefix

It is the start text of wordN.

  wordN_suffix

It is the end text of wordN.

  Return value

It returns snippet string.

  mroonga_snippet_html()

Added in version 5.09.

  Summary

mroonga_snippet_html() UDF provides functionality to get highlighted keyword in context.

  Syntax

mroonga_snippet_html() has required parameter and optional parameter:

    mroonga_snippet_html(document, key_word)

  Usage

  Parameters

  Required parameters

There are many required parameters.

  Optional parameters

  Return value

It returns snippet string.

  Limitations

There are some limitations in Mroonga storage engine.

  Limitations of table

A table has the following limitations. This limitation is derived from Groonga.

•   The maximum one key size: 4KiB

•   The maximum total size of keys: 4GiB

•   The maximum number of records:

•   No primary key table: 1,073,741,815 (2 30 - 9)

•   PRIMARY KEY or PRIMARY KEY USING BTREE table: 1,073,741,823 (2 30 - 1)

•   PRIMARY KEY USING HASH table: 536,870,912 (2 29)

    Keep in mind that these limitations may vary depending on conditions.

  Limitations of indexing

A full-text index has the following limitations. This limitation is derived from groonga and applied to each table.

•   The maximum number of distinct terms: 268,435,455 (more than 268 million)

•   The maximum index size: 256GiB

    Keep in mind that these limitations may vary depending on conditions.

    Confirm by SHOW TABLE STATUS or SHOW INDEX FROM ... whether your table data matches to this limitations.

  Limitations about the primary key

Empty data (0 byte data) is not allowed in the primary key.

  Limitations about the value of columns

There is a limitation about the value of column in storage mode.

Mroonga storage engine executes automatic conversion against the value NULL.

For example, if the value NULL is used in DATE or DATETIME columns, Mroonga storage engine automatically converts 0 into 1 as the value of month or date.

Thus, the value 0 is treated as the 1st month (January) of the year or the 1st date of the month.

And more, the value NULL is treated as the value of UNIX time 0 (1970-01-01 00:00:00).

This kind of automatic conversion is not restricted to only DATE or DATETIME types.

The value NULL is converted into the default value of columns. In most cases, it will be converted into empty string for column which belongs to type of string, 0 for column which belongs to type of numeric.

Here is an example to show behavior described above.

    CREATE TABLE date_limitation (
      id INT PRIMARY KEY AUTO_INCREMENT,
      input varchar(32) DEFAULT NULL,
      date DATE DEFAULT NULL
    ) ENGINE=mroonga DEFAULT CHARSET=UTF8;
    CREATE TABLE datetime_limitation (
      id INT PRIMARY KEY AUTO_INCREMENT,
      input varchar(32) DEFAULT NULL,
      datetime DATETIME DEFAULT NULL
    ) ENGINE=mroonga DEFAULT CHARSET=UTF8;
    -- Test data for date_limitation
    INSERT INTO date_limitation (input) VALUES ("NULL");
    INSERT INTO date_limitation (input, date) VALUES ("1970-00-00", "1970-00-00");
    -- Test data for datetime_limitation
    INSERT INTO datetime_limitation (input) VALUES ("NULL");
    INSERT INTO datetime_limitation (input, datetime) VALUES ("1970-00-00 00:00:00", "1970-00-00 00:00:00");

Here is the results of execution example:

    mysql> select * from date_limitation;
    +----+------------+------------+
    | id | input      | date       |
    +----+------------+------------+
    |  1 | NULL       | 1970-01-01 |
    |  2 | 1970-00-00 | 1970-01-01 |
    +----+------------+------------+
    2 rows in set (0.00 sec)
    mysql> select * from datetime_limitation;
    +----+---------------------+---------------------+
    | id | input               | datetime            |
    +----+---------------------+---------------------+
    |  1 | NULL                | 1970-01-01 00:00:00 |
    |  2 | 1970-00-00 00:00:00 | 1970-01-01 00:00:00 |
    +----+---------------------+---------------------+
    2 rows in set (0.00 sec)

  Limitations of column size

A column has the following limitation.

•   The maximum stored data size of a column: 256GiB

  Full text search

  Boolean mode

  Summary

Mroonga can perform boolean full text searches using the IN BOOLEAN MODE modifier for MATCH AGAINST:

    SELECT ... WHERE MATCH(column) AGAINST ('...' IN BOOLEAN MODE);

Normally, IN BOOLEAN MODE is suitable rather than the default IN NATURAL LANGUAGE MODE. Because IN BOOLEAN MODE is similar to query in Web search engine. Most people familiar with query in Web search engine.

You can use qualifiers which MySQL support <https://dev.mysql.com/doc/refman/5.7/en/fulltext-boolean.html> and Mroonga original pragmas in boolean full text search query.

These qualifiers and pragmas can change the relative rank of search results.

In the case of a search string not using neither a qualifier nor a pragma, the search results that contain the search string will be rated higher.

  Usage

Here are schema and data to show examples:

    CREATE TABLE books (
      `id` INTEGER AUTO_INCREMENT,
      `title` text,
      PRIMARY KEY(`id`),
      FULLTEXT INDEX title_index (title)
    ) ENGINE=Mroonga DEFAULT CHARSET=utf8mb4;
    INSERT INTO books (title) VALUES ('Professional MySQL');
    INSERT INTO books (title) VALUES ('MySQL for Professional');
    INSERT INTO books (title) VALUES ('Mroonga = MySQL + Groonga');

  Qualifier

Here are supported qualifiers.

  KEYWORD1 KEYWORD2

No operator between keywords such as KEYWORD1 KEYWORD2 indicates that one of keywords must be present in each row that is returned.

Mroonga for query means that Mroonga or for must be present:

    SELECT title
      FROM books
     WHERE MATCH(title) AGAINST('Mroonga for' IN BOOLEAN MODE);
    -- +---------------------------+
    -- | title                     |
    -- +---------------------------+
    -- | Mroonga = MySQL + Groonga |
    -- | MySQL for Professional    |
    -- +---------------------------+

  KEYWORD1 OR KEYWORD2

OR (must be uppercase) indicates that left hand side keyword or right hand side keyword must be present in each row that is returned.

Mroonga OR for query means that Mroonga or for must be present:

    SELECT title
      FROM books
     WHERE MATCH(title) AGAINST('Mroonga OR for' IN BOOLEAN MODE);
    -- +---------------------------+
    -- | title                     |
    -- +---------------------------+
    -- | Mroonga = MySQL + Groonga |
    -- | MySQL for Professional    |
    -- +---------------------------+

OR is the default operator. You can omit it. Both Mroonga OR for and Mroonga for return the same result.

  +KEYWORD

A leading plus sign indicates that this word must be present in each row that is returned.

+MySQL +Mroonga query means that both MySQL and Mroonga must be present:

    SELECT title
      FROM books
     WHERE MATCH(title) AGAINST('+MySQL +Groonga' IN BOOLEAN MODE);
    -- +---------------------------+
    -- | title                     |
    -- +---------------------------+
    -- | Mroonga = MySQL + Groonga |
    -- +---------------------------+

  -KEYWORD

A leading minus sign indicates that this word must not be present in any of the rows that are returned.

+MySQL -Mroonga query means that MySQL must be present but Mroonga must not be present:

    SELECT title
      FROM books
     WHERE MATCH(title) AGAINST('+MySQL -Mroonga' IN BOOLEAN MODE);
    -- +------------------------+
    -- | title                  |
    -- +------------------------+
    -- | Professional MySQL     |
    -- | MySQL for Professional |
    -- +------------------------+

  PREFIX*

A following asterisk indicates that all words starting with this word must be present in any of the rows that are returned.

+M* query means that words starting M (MySQL and Mroonga in this case) must be present:

    SELECT title
      FROM books
     WHERE MATCH(title) AGAINST('+M*' IN BOOLEAN MODE);
    -- +---------------------------+
    -- | title                     |
    -- +---------------------------+
    -- | Mroonga = MySQL + Groonga |
    -- | Professional MySQL        |
    -- | MySQL for Professional    |
    -- +---------------------------+

Note: To be precise, "word" may not be "word" you think. "word" in this context is "token". "token" may not be word. For example, tokens in "It's" are "It", "'" and "s".

You can confirm token by mroonga_command() <> and tokenize <https://groonga.org/docs/reference/commands/tokenize.html>:

    SELECT mroonga_command('tokenize TokenBigram "It''s" NormalizerMySQLGeneralCI');
    -- +--------------------------------------------------------------------------+
    -- | mroonga_command('tokenize TokenBigram "It''s" NormalizerMySQLGeneralCI') |
    -- +--------------------------------------------------------------------------+
    -- | [                                                                        |
    -- |   {                                                                      |
    -- |     "value":"IT",                                                        |
    -- |     "position":0,                                                        |
    -- |     "force_prefix":false                                                 |
    -- |   },                                                                     |
    -- |   {                                                                      |
    -- |     "value":"'",                                                         |
    -- |     "position":1,                                                        |
    -- |     "force_prefix":false                                                 |
    -- |   },                                                                     |
    -- |   {                                                                      |
    -- |     "value":"S",                                                         |
    -- |     "position":2,                                                        |
    -- |     "force_prefix":false                                                 |
    -- |   }                                                                      |
    -- | ]                                                                        |
    -- +--------------------------------------------------------------------------+

JSON value in the above result is formatted by hand.

  "PHRASE"

Quoting phrase by double quote (") indicates that the phrase must be present in any of the rows that are returned.

+"Professional MySQL" query means that Professional MySQL phrase must be present. The query doesn't match to MySQL for Profession. MySQL for Profession includes both MySQL and Professional words but doesn't include Professional MySQL phrase:

    SELECT title
      FROM books
     WHERE MATCH(title) AGAINST('+"Professional MySQL"' IN BOOLEAN MODE);
    -- +--------------------+
    -- | title              |
    -- +--------------------+
    -- | Professional MySQL |
    -- +--------------------+

  (SUBEXPRESSION...)

Parentheses groups expressions.

+(Groonga OR Mroonga) +MySQL query means the following:

•   Groonga or Mroonga must be present.

•   MySQL must be present.

    Here is the result of the query:

        SELECT title
          FROM books
         WHERE MATCH(title) AGAINST('+(Groonga OR Mroonga) +MySQL' IN BOOLEAN MODE);
        -- +---------------------------+
        -- | title                     |
        -- +---------------------------+
        -- | Mroonga = MySQL + Groonga |
        -- +---------------------------+

  Pragma

Pragma is metadata for query. You can change how to parse query by specifying pragma.

You can embed pragma at the head of query for specifying how to execute.

Pragma must exist in the beginning of a query. Don't put a blank into a head of the query. Pragma starts with *:

    SELECT MATCH AGAINST('*PRAGMA ...' IN BOOLEAN MODE);

You can specify multiple pragmas:

    SELECT MATCH AGAINST('*PRAGMA1PRAGMA2 ...' IN BOOLEAN MODE);

Here are available pragmas.

  D pragma

D pragma indicates the default operator. It's used when an individual operator is omitted.

Here is the D pragma syntax. You can choose one of OR, + or - as ${OPERATOR}:

    *D${OPERATOR}

  DOR

DOR means that "or" is used as the default operator.

This is the default.

Here is an example to use DOR. '*DOR for Mroonga' IN BOOLEAN MODE returns records that includes for or Mroonga:

    SELECT title
     FROM books
    WHERE MATCH (title) AGAINST('*DOR for Mroonga' IN BOOLEAN MODE);
    -- +---------------------------+
    -- | title                     |
    -- +---------------------------+
    -- | MySQL for Professional    |
    -- | Mroonga = MySQL + Groonga |
    -- +---------------------------+

  D+

D+ means that "and" is used as the default operator. It's similar to query in Web search engine.

Here is an example to use D+. '*D+ MySQL Mroonga' IN BOOLEAN MODE returns records that includes MySQL and Mroonga:

    SELECT title
     FROM books
    WHERE MATCH (title) AGAINST('*D+ MySQL Mroonga' IN BOOLEAN MODE);
    -- +---------------------------+
    -- | title                     |
    -- +---------------------------+
    -- | Mroonga = MySQL + Groonga |
    -- +---------------------------+

  D-

D- means that "not" is used as the default operator.

Here is an example to use D-. '*D- MySQL Mroonga' IN BOOLEAN MODE returns records that includes MySQL but doesn't include Mroonga:

    SELECT title
     FROM books
    WHERE MATCH (title) AGAINST('*D- MySQL Mroonga' IN BOOLEAN MODE);
    -- +------------------------+
    -- | title                  |
    -- +------------------------+
    -- | Professional MySQL     |
    -- | MySQL for Professional |
    -- +---------------------------+

  W pragma

W pragma indicates target section and its weight for multiple column index.

You can specify different weight for each section. The default weight is 1. 1 means that no weight.

Here is the W pragma syntax. ${SECTION} is a number that is begun not from 0 but from 1. ${WEIGHT} is omitable:

    *W[${SECTION1}[:${WEIGHT1}]][,${SECTION2}[:${WEIGHT2}]][,...]

Here are schema and data to show examples. You need to create a multiple column index to use W pragma:

    CREATE TABLE memos (
      `id` INTEGER AUTO_INCREMENT,
      `title` text,
      `content` text,
      PRIMARY KEY(`id`),
      FULLTEXT INDEX text_index (title, content)
    ) ENGINE=Mroonga DEFAULT CHARSET=utf8mb4;
    INSERT INTO memos (title, content) VALUES (
      'MySQL', 'MySQL is a RDBMS.'
    );
    INSERT INTO memos (title, content) VALUES (
      'Groonga', 'Groonga is a full text search engine.'
    );
    INSERT INTO memos (title, content) VALUES (
      'Mroonga', 'Mroonga is a storage engine for MySQL based on Groonga.'
    );

Here is an example to show how to use weight. title column has 10 weight and content columns has 1 weight. It means that keyword in title column is 10 times important than keyword in content column:

    SELECT title,
           content,
           MATCH (title, content) AGAINST('*W1:10,2:1 +Groonga' IN BOOLEAN MODE) AS score
      FROM memos;
    -- +---------+--------------------------------------------------------+-------+
    -- | title   | content                                                | score |
    -- +---------+--------------------------------------------------------+-------+
    -- | MySQL   | MySQL is a RDBMS.                                      |     0 |
    -- | Groonga | Groonga is a full text search engine.                  |    11 |
    -- | Mroonga | Mroonga is a storage engine for MySQL based on Groonga |     1 |
    -- +---------+--------------------------------------------------------+-------+

The score of the first record is 0. Because it doesn't have any Groonga in both title column and content column.

The score of the second record is 11. Because it has Groonga in both title column and content column. Groonga in title column has score 10. Groonga in content column has score 1. 11 is sum of them.

The score of the third record is 1. Because it has Groonga in only content column. `Groonga in content column has score 1. So the score of the record is 1.

  S pragma

S pragma indicates syntax of the query.

Here is a syntax of S pragma:

    *S${SYNTAX}

Here is a list of available syntax:

•   S: Script syntax <https://groonga.org/docs/reference/grn_expr/script_syntax.html>

  *SS

You can use script syntax <https://groonga.org/docs/reference/grn_expr/script_syntax.html> by *SS pragma. You can use full Groonga search features in script syntax.

Here are schema and data to show example of script syntax usage:

    CREATE TABLE comments (
      `content` text,
      FULLTEXT INDEX content_index (content)
    ) ENGINE=Mroonga DEFAULT CHARSET=utf8mb4;
    INSERT INTO comments VALUES (
      'A student started to use Mroonga storage engine. It is very fast!'
    );
    INSERT INTO comments VALUES (
      'Another student also started to use Mroonga storage engine. It is very fast!'
    );

Here is an example to use near search <https://groonga.org/docs/reference/grn_expr/script_syntax.html#near-search-operator> by script syntax:

    SELECT content,
           MATCH (content) AGAINST('*SS content *N "student fast"' IN BOOLEAN MODE) AS score
      FROM comments;
    -- +------------------------------------------------------------------------------+-------+
    -- | content                                                                      | score |
    -- +------------------------------------------------------------------------------+-------+
    -- | A student started to use Mroonga storage engine. It is very fast!            |     1 |
    -- | Another student also started to use Mroonga storage engine. It is very fast! |     0 |
    -- +------------------------------------------------------------------------------+-------+

Near search matches only when there are 10 or less words between specified words (student and fast in this case). So student started ...(8 words)... very fast is matched but student also started ...(8 words)... very fast isn't matched.

You can also use other advanced features.

  Search and Scoring in Mroonga

  Mroonga Scoring in natural language mode

The search score of Mroonga in natural language mode is a similarity score between query and document. Mroonga's scoring algorithm is as follows:

1.  Splitting a query into tokens.

2.  Removing not matched tokens.

3.  Calculating weight per token.

4.  Getting the top N weight tokens.

5.  Summing up the weight per token which exists in a document, and is one of the top N weight tokens. The total weight is the similarity score for the document and the query.

    We are showing by example. The first thing we create a table and insert data as follows:

        SET NAMES UTF8;
        CREATE TABLE diaries (
          id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
          content TEXT,
          FULLTEXT INDEX(content)
        ) ENGINE mroonga DEFAULT CHARSET UTF8;
        INSERT INTO diaries (content) VALUES("It'll be fine tomorrow as well.");
        INSERT INTO diaries (content) VALUES("It'll rain tomorrow.");
        INSERT INTO diaries (content) VALUES("It's fine today. It'll be fine tomorrow as well.");
        INSERT INTO diaries (content) VALUES("It's fine today. But it'll rain tomorrow.");

    We use a query "fine today" to search. The search result is as follows:

        mysql> SELECT *, MATCH (content) AGAINST ("fine today") AS score
            ->        FROM diaries
            ->        WHERE MATCH (content) AGAINST ("fine today")
            ->        ORDER BY MATCH (content) AGAINST ("fine today") DESC;
        +----+--------------------------------------------------+--------+
        | id | content                                          | score  |
        +----+--------------------------------------------------+--------+
        |  3 | It's fine today. It'll be fine tomorrow as well. | 131073 |
        |  4 | It's fine today. But it'll rain tomorrow.        | 131073 |
        +----+--------------------------------------------------+--------+
        2 rows in set (0.01 sec)

    Now, let us explain how to make the result score "131073".

  Splitting a query into tokens

The default tokenizer splits up the query "fine today" into two tokens as follows:

•   fine

•   today

  Removing not matched tokens

We have nothing to do in this case because the tokens exist in documents.

•   fine: include in document id=1,3,4

•   today: include in document id=3,4

  Calculating weight per token

•   fine: 116508(= 1048576 / 9)

•   today: 131072(= 1048576 / 8)

    1048576(= 2 ** 20) denotes the number of tokens in all documents. We should get the number from the table, but as a matter of fact, the fixed value is used for simplicity.

    In the fraction 1048576 / 8, the denominator "8" indicates that the number of document which contains token "today".

    In the fraction 1048576 / 9, the denominator "9" means the number of document which contains token "fine".

    The denominator "8" and "9" are approximate number, strictly speaking, "2" is the correct answer for the number of document that contains "today", "3" is the correct answer for the number of document that contains "fine".

    You can check the approximate number of token by using the following query.

        SELECT mroonga_command("select diaries-content --query '_key:fine OR _key:today' --output_columns _key, index --limit -1") AS groonga_response;

    The retrieval result of the above query is as follows:

        [[[2],[["_key","ShortText"],["index","diaries"]],["FINE",9],["TODAY",8]]]

  Getting the top N weight tokens

The formula for the N is "the number of occurrence of token in document / 8 + 1"

In this case, N = 2 / 8 + 1 ≒ 1

The sorted tokens are as follows:

•   today: 131072(= 1048576 / 8)

•   fine: 116508(= 1048576 / 9)

    N = 1, then we get "today: 131072(= 1048576 / 8)".

  Summing up the weight per token which exists in a document, and is one of the top N weight tokens

•   today: include in document id=3,4

    Finally document id 3,4 are hit, the similarity score between query and document(id=3) is 131072 + 1 = 131073 ("1" is the number of occurrence of token "today" in document).

    The similarity score between query and document(id=4) is the same as the score between query and document(id=3).

FAQ

There are some frequently asked questions for Mroonga. This section describes about them.

  How to avoid "mmap Cannot allocate memory" error?

  Short answer

Increase vm.max_map_count Linux kernel parameter value.

  Long answer

There is a case following mmap error in log file:

    2013-06-04 08:19:34.835218|A|4e86e700|mmap(4194304,551,432017408)=Cannot allocate memory <13036498944>

Note that <13036498944> means total size of mmap (almost 12GB) in this case.

So you need to confirm following point of views.

•   Are there enough free memory?

•   Are maximum number of mappings exceeded?

    To check there are enough free memory, you can use vmstat command.

    To check whether maximum number of mappings are exceeded, you can investigate the value of vm.max_map_count.

    If this issue is fixed by modifying the value of vm.max_map_count, it's exactly the reason.

    As Groonga allocates memory chunks each 256KB, you can estimate the size of database you can handle by following formula:

        (database size) = vm.max_map_count * (memory chunks)

    If you want to handle over 16GB Groonga database, you must specify at least 65536 as the value of vm.max_map_count:

        database size (16GB) = vm.max_map_count (65536) * memory chunks (256KB)

    You can modify vm.max_map_count temporary by sudo sysctl -w vm.max_map_count=65536.

    Then save the configuration value to /etc/sysctl.conf or /etc/sysctl.d/*.conf.

COMMUNITY

There are some places for sharing Groonga and Mroonga information. We welcome you to join our community.

  Mailing List

There are mailing lists for discussion about Groonga and Mroonga.

For English speakers
    groonga-talk@lists.sourceforge.net <http://lists.sourceforge.net/mailman/listinfo/groonga-talk>

For Japanese speakers
    groonga-dev@lists.osdn.me <http://lists.osdn.me/mailman/listinfo/groonga-dev>

  Chat room

There are chat rooms for discussion about Groonga and Mroonga.

For English speakers
    groonga/en chat room on Gitter <https://gitter.im/groonga/en>

For Japanese speakers
    groonga/ja chat room on Gitter <https://gitter.im/groonga/ja>

  Twitter

@groonga <https://twitter.com/groonga/> tweets Groonga and Mroonga related information.

Please follow the account to get the latest Groonga and Mroonga related information!

  Facebook

Mroonga page on Facebook <http://www.facebook.com/mroonga> shares Mroonga related information.

Please like the page to get the latest Mroonga related information!

HOW TO CONTRIBUTE TO MROONGA

We welcome your contributions to the Mroonga Project. There are many ways to contribute, such as using Mroonga, introduction to others, etc. For example, if you find a bug when using Mroonga, you are welcome to report the bug. Coding and documentation are also welcome for Mroonga and its related projects.

As a user:
    If you are interested in Mroonga, please read this document and try it.

As a spokesman:
    Please introduce Mroonga to your friends and colleagues.

As a developer: Bug report, development and documentation
    This section describes the details.

  How to report a bug

There are some ways to report a bug.

You can use either way It makes no difference to us.

  Submit a bug to the issue tracker

Mroonga project uses GitHub issue tracker <https://github.com/mroonga/mroonga/issues>.

You can use English or Japanese to report a bug.

  Report a bug with the other ways

Mroonga project has Community <> for discussing about Mroonga. Please send an Email that describes a bug.

  How to collect data for reporting a reproducible bug

There may be a case that Mroonga doesn't work as expected. It may be a crash bug or SQL returns unexpected search results. To solve such a issues with you, We need a reproducible bug report.

For a reproducible bug report, it is better to report with the following information as much as you can because it is useful to investigate the reason which causes the bug.
Name | Description
Environment information | Installed packages information to detect what version is used. rpm: rpm -qa > INSTALLED.txt deb: dpkg -l > INSTALLED.txt On Windows, show us what zip package is used.
Executing query which triggers the issue | It shows what you did.
Sample schema and data | It is recommended to provide a minimum schema and sample data to reproduce the issue. It helps to solve the issue sooner.
MySQL error log | It may include backtrace information why crashed unexpectedly. It may exists on /var/log/mysqld.log for example. (It depends on log-error configuration)
groonga.log | It may include low level error information. It may exists on /var/lib/mysql/groonga.log for example. (It depends on mroonga_log_file configuration)

There is a case that it is not enough to reproduce a bug with above information. For such a case, additional information is required.
Name | Description
MySQL dump | It may be required to construct same database from scratch. It may be generated by mysqldump -u USER -pPASSWORD --opt --flush-logs --single-transaction --master-data=2 --default-character-set=utf8 --hex-blob --databases DATABASE > dump.sql and so on.
MySQL binlog | If dump data is enough to reproduce the issue, it is not required. It may exists on /var/lib/mysql/mysql-bin.* for example. (It depends on log-bin configuration)
MySQL data directory (Offline backup) | If dump data and binlog pair is lost, It may be required. It may exists on /var/lib/mysql/ for example. (It depends on datadir configuration)

  How to contribute in documentation topics

We use Sphinx for documentation tool.

具体的にどんなことをやればいいのかを説明します。基本的にはドキュメントのソースファイルごとにpull requestを送ってもらうと進めやすいです。

対象となるファイルはdoc/sourceディレクトリ以下の拡張子が「.rst」となっているファイルです。

あまりGitHubでの作業に慣れていなくてもできるように、「最初にやること」と「作業ごとにやること」、「ファイルごとにやること」に分けて順に説明します。

•   The things you must do at first

•   The things you need to do every tasks

•   The things you need to do every files

  The things you must do at first

以下では、最初に一度だけ実施しておけば良いことを説明します。

  Git configuration

まずは、gitの設定をしましょう。すでにある程度gitを使っている場合には初期設定はすでに完了しているかも知れません。その場合には飛ばして構いません。

    $ git config --global user.name "Your Name"
    $ git config --global user.email "Email address"

上記はコミットログに使われます。公開しても差し支えないユーザ名もしくはメールアドレスを設定します。

  Fork on GitHub

First, create GitHub account. If your GitHub account is ready, login to GitHub and access following URL.

•   Fork the Mroonga repository <https://github.com/mroonga/mroonga/fork>

    Fork リポジトリ選択画面でご自分のリポジトリへとforkしてください。

  Initial configuration for working repository

Clone Mroonga repository to working directory. Don't forget to do "Git configuration".

    $ git clone git@github.com:(YOUR_GITHUB_ACCOUNT)/mroonga.git
    $ cd mroonga
    $ git remote add upstream git@github.com:mroonga/mroonga

  Prerequisites

Before generating the Mroonga documentation, we need to build Mroonga. Please refer to Others <> for the instructions on how to build Mroonga.

  Required software

Generating Mroonga documentation needs the followings.

•   Sphinx <https://www.sphinx-doc.org/>

•   gettext gem <https://github.com/ruby-gettext/gettext>

    We use Sphinx for documentation tool and use gettext gem for localization. You can install both tools using the following commands.

        $ pip install -r doc/requirements.txt
        $ (cd doc && bundle install)

  Initial configuration for building documentation

Execute following commands to prepare for generating Mroonga documentation.

    $ cmake \
        -S . \
        -B ../mroonga.doc \
        --preset=doc \
        -DMYSQL_SOURCE_DIR=(MySQL_SOURCE_DIRECTORY) \
        -DMYSQL_BUILD_DIR=(MySQL_BUILD_DIRECTORY) \
        -DMYSQL_CONFIG=(MySQL_CONFIG)

Next step is "The things you need to do every tasks".

  The things you need to do every tasks

以下では作業ごとにやることを説明します。

  Follow the upstream

Mroonga本家の最新状態に追従して、作業がかぶらないようにします。

    $ git fetch --all
    $ git checkout main
    $ git rebase upstream/main

最新の状態に追従できたら、「ファイルごとにやること」へと進みます。

  The things you need to do every files

以下では、例えば The characteristics of Mroonga <> を更新する場合で説明します。作業対象となるファイルは、リポジトリのdoc/source/ディレクトリ以下にあり拡張子が.rstなファイルです。今回は、doc/source/characteristic.rstを変更する例で説明します。

  Create working branch

Create a working branch. Use meaningful branch name.

    $ git checkout -b use-capitalized-notation-characteristic

  Editing text

Fix typos, styles or write a new document for Mroonga.

  Confirm generated document

Execute following command to generate HTML files that reflect your changes.

    $ cmake --build ../mroonga.doc

Open the generated file in your Web browser to preview your changes are reflected.

    $ open ../mroonga.doc/doc/en/html/characteristic.html

  Commit

HTMLに問題がないことを確認できたら、コミットします。

    $ cd ${cloneしたディレクトリーのトップディレクトリー}
    $ git add doc/source/characteristic.rst
    $ git commit

コミットするときのメッセージについては、例えば以下のようにします。

    doc: use "Mroonga" notation

  Push and pull request

Publish your changes to your own GitHub repository.

    $ git push -u origin use-capitalized-notation-characteristic

Note that use-capitalized-notation-characteristic is already created branch in advance.

ブラウザで https://github.com/(GitHubのアカウント)/mroonga を開くと「 @use-capitalized-notation-characteristic@ 」ブランチをpull requestする！みたいなUIができているので、そこのボタンを押してpull requestしてください。入力フォームがでてきますが、コミットしたときメッセージで十分なのでそのままpull requestしてOKです！

これで、ひととおりの作業は完了しました。

DEVELOPER'S GUIDE

  How to debug

  Building for debugging

When you build software for debugging, you can get more information like symbol resolutions in GDB. So we build both MySQL and Mroonga for debugging in development.

Note: If you build one of them for debugging, the size of structures etc. might be different, and you might not be able to load Mroonga, or assertions don't work in running.

  How to build MySQL for debugging

As you can see in MySQL :: 2.9.4 Installing MySQL Using a Standard Source Distribution <https://dev.mysql.com/doc/refman/8.4/en/installing-source-distribution.html>, you can build MySQL for debugging by passing -DWITH_DEBUG=ON option in CMake options.

The procedure from download to build is the following:

    $ mkdir -p ~/work/
    $ cd ~/work/
    $ wget https://cdn.mysql.com/Downloads/MySQL-8.4/mysql-8.4.2.tar.gz
    $ tar xf mysql-8.4.2.tar.gz
    $ cmake \
        -Smysql-8.4.2 \
        -Bmysql-8.4.2.build \
        -GNinja \
        -DCMAKE_BUILD_TYPE=Debug \
        -DCMAKE_INSTALL_PREFIX=/tmp/local \
        -DWITH_DEBUG=ON
    $ cmake --build mysql-8.4.2.build

  How to build MariaDB for debugging

If you want to use MariaDB instead of MySQL, you can use -DWITH_DEBUG=ON too.

Note that you need to remove storage/mroonga/ (Mroonga bundled in MariaDB) before you build MariaDB.

The procedure from download to build is the following:

    $ mkdir -p ~/work/
    $ cd ~/work/
    $ release_id=$(curl -s https://downloads.mariadb.org/rest-api/mariadb/ \
        | jq -r '.major_releases[] | select(.release_support_type == "Long Term Support" and .release_status == "Stable") | .release_id' \
        | head -n1)
    $ version=$(curl -s https://downloads.mariadb.org/rest-api/mariadb/$release_id \
        | jq -r '.releases[].release_id' \
        | head -n1)
    $ wget https://downloads.mariadb.org/rest-api/mariadb/$version/mariadb-$version.tar.gz
    $ tar xf mariadb-$version.tar.gz
    $ rm -rf mariadb-$version/storage/mroonga
    $ cmake \
        -Smariadb-$version \
        -Bmariadb-$version.build \
        -GNinja \
        -DCMAKE_BUILD_TYPE=Debug \
        -DCMAKE_INSTALL_PREFIX=/tmp/local \
        -DWITH_DEBUG=ON
    $ cmake --build mariadb-$version.build

  How tom build Mroonga for debugging

You can build Mroonga for debugging by passing --preset=debug in CMake options.

Note: In order to build Mroonga, you need to install required tools and libraries beforehand.

See Others <> for the details of dependencies.

The procedure from cloning repository to build is the following:

    $ cd ~/work/
    $ git clone git@github.com:mroonga/mroonga.git
    $ cmake \
        -Smroonga \
        -Bmroonga.mysql-8.4 \
        --preset=debug \
        -DCMAKE_INSTALL_PREFIX=/tmp/local \
        -DMYSQL_BUILD_DIR=$HOME/work/mysql-8.4.2.build \
        -DMYSQL_CONFIG=$HOME/work/mysql-8.4.2.build/scripts/mysql_config \
        -DMYSQL_SOURCE_DIR=$HOME/work/mysql-8.4.2
    $ cmake --build mroonga.mysql-8.4

When you successfully build both, please invoke tests like the following. If you get [pass] for all tests, you succeeded to build for debugging:

    $ cd mroonga.mysql-8.4
    $ ../mroonga/test/run-sql-test.sh

  More about run-sql-test.sh

run-sql-test.sh is our friend for debugging. Here we show some examples of its usage.

  Run the specified test only

When you invoke run-sql-test.sh without any option, all tests under mysql-test/mroonga will be invoked.

So if you want to run certain tests only, you can specify the test:

    $ ../mroonga/test/run-sql-test.sh ../mroonga/mysql-test/mroonga/storage/t/truncate.test

  See the trace

When you run tests by adding --debug option like the following, function calls information is recorded:

    $ ../mroonga/test/run-sql-test.sh --debug ../mroonga/mysql-test/mroonga/storage/t/truncate.test

This information is stored in ${MySQL's build directory}/mysql-test/var/log/mysqld.1.trace.

When you add a new method, it would be a good idea to put MRN_DBUG_ENTER_METHOD()/DBUG_RETURN() in the beginning/ending of method and record its calls.

  Invoking GDB

By adding --manual-gdb option, you can debug with GDB when you run tests:

    $ ../mroonga/test/run-sql-test.sh --manual-gdb ../mroonga/mysql-test/mroonga/storage/t/truncate.test

You need to run gdb in other terminal for this. The command line to run gdb will be showed by run-sql-test.sh.

  Release procedure

  Requirements

Here is the requirements about release procedure.

•   Debian GNU/Linux (sid)

•   Use zsh as user shell

    Use the following working directories.

•   MROONGA_SOURCE_DIR=$HOME/work/mroonga

•   MROONGA_BUILD_DIR=$HOME/work/build-dir/mroonga

•   MROONGA_ORG_DIR=$HOME/work/mroonga.org

•   MARIADB_SOURCE_DIR=$HOME/work/mariadb

•   MARIADB_BUILD_DIR=$HOME/work/build-dir/mariadb

•   GROONGA_SOURCE_DIR=$HOME/work/groonga

•   GROONGA_BUILD_DIR=$HOME/work/build-dir/groonga

•   INSTALL_DIR=/tmp/local

  Setup build environment

Install the following packages:

    $ sudo apt-get install -V ruby mecab libmecab-dev gnupg2 dh-autoreconf bison

  Describe the changes

Summarize recent changes since the previous release into doc/source/news/xx.md.

Should be included

•   The changes which affect to users

•   The changes which broke compatibility

    Shoud not be included

•   The changes which doesn't affect to users (Internal source code changes or refactoring)

    Execute the following command to create HTML for news:

        $ cmake -S ${MARIADB_SOURCE_DIR} -B ${MARIADB_BUILD_DIR} -GNinja -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -DPLUGIN_CASSANDRA=NO
        $ cmake --build ${MARIADB_BUILD_DIR}
        $ cmake --install ${MARIADB_BUILD_DIR}
        $ cmake -S ${GROONGA_SOURCE_DIRECTORY} -B ${GROONGA_BUILD_DIR} --preset=release-default --fresh -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}"
        $ cmake --build ${GROONGA_BUILD_DIR}
        $ cmake --install ${GROONGA_BUILD_DIR}
        $ PKG_CONFIG_PATH=${INSTALL_DIR}/lib/pkgconfig cmake -S ${MROONGA_SOURCE_DIRECTORY} -B ${MROONGA_BUILD_DIR} --fresh --preset=doc -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" -DMYSQL_SOURCE_DIR=~${MARIADB_SOURCE_DIR} -DMYSQL_BUILD_DIR=${MARIADB_BUILD_DIR} -DMYSQL_CONFIG=${INSTALL_DIR}/bin/mariadb_config
        $ cmake --build ${MROONGA_BUILD_DIR}

  Execute release task

Execute rake release:

    $ rake release

  About release task

release task executes the three tasks.

1.  release:version:update

•   Append the changelog of the new version to the spec file of the RPM package, and so on

•   You can also specify a release date with NEW_RELEASE_DATE

2.  release:tag

•   Push the tag for release

•   This will start the automatic release

3.  dev:version:bump

•   We will bump up the version for the next release

•   You can also specify a version with NEW_VERSION

  Confirm CI

We confirm whether below CIs green or not.

•   GitHub Actions <https://github.com/mroonga/mroonga/actions>

    We use CI to do automatic releases, so if it fails, we retry.

  Update Docker images

Execute the following commands:

    $ git clone git@github.com:mroonga/docker.git mroonga-docker
    $ cd mroonga-docker
    $ ./update.sh
    $ git push
    $ git push --tags

  Update documents

Execute the following commands:

    $ export GROONGA_ORG_REPOSITORY=${HOME}/work/groonga.org
    $ git clone git@github.com:groonga/groonga.org.git ${GROONGA_ORG_REPOSITORY}
    $ rake -C ${MROONGA_ORG_DIR} release

  Announce release for X(Twitter)

Click Tweet link in Mrooga blog entry. You can share tweet about latest release. If you use tweet link, title of release announce and URL is embedded into your tweet.

Execute sharing tweet in Japanese and English version of blog entry. Note that this tweet should be done when logged in by groonga account.

  Announce release for Facebook

We announce release from Mroonga group in Facebook.

<https://www.facebook.com/mroonga/> TODO: Translate.

  コーディングスタイル

一般的に1つのコードベースに複数のスタイルがまざっているとソースコードが読みづらくなります。たとえ、それぞれのスタイル単独では読みやすいスタイルあっても、まざると読みづらくなります。そのため、Mroongaプロジェクトでもスタイルを統一して、読みやすいソースコードになるようにします。

読みやすいソースコードにする理由は以下の通りです。

•   機能を拡張するときのコストを下げる。

•   問題を修正するときのコストを下げる。

    どちらの場合も周辺のソースコードを読んで、それをベースにコードを追加・変更します。このとき、ソースコードが読みやすい状態だと周辺のソースコードの把握をスムーズに行うことができ、スムーズにその後の作業に移れます。

    TODO: 読みやすさの他にデバッグのしやすさ（gdbでの追いやすさ）も考慮に入れたほうがよさそうだがどうしよう。

  言語

基本的にすべてC++で記述します。よほどのことがない限りCは使いません。

よい例:

    ha_mroonga.cpp

悪い例（C言語を使っている）:

    mrn_sys.c

  ファイル名

ソースコードのファイル名は全て小文字にします。また、単語ごとに"_"で区切ります。

よい例:

    ha_mroonga.cpp

悪い例（大文字を使っている）:

    HA_MROONGA.cpp

悪い例（単語を"_"で区切らずにくっつけている）:

    hamroonga.cpp

悪い例（単語を"-"で区切っている）:

    ha-mroonga.cpp

ソースコードの拡張子 .cpp にします。

よい例:

    ha_mroonga.cpp

悪い例（ .cc を使っている）:

    ha_mroonga.cc

ヘッダーファイルの拡張子は .hpp にします。

よい例:

    ha_mroonga.hpp

悪い例（ .h を使っている）: ha_mroonga.h

  名前空間

ヘッダーファイルでは using namespace を使わない。ソースコードでは using namespace std であれば使ってもよい。他の名前空間は使ってはいけない。

よい例:

    ha_mroonga.cpp:
      using namespace std;

悪い例（ヘッダーファイル内で using namespace を使っている）:

    ha_mroonga.hpp:
      using namespace std;

悪い例（ std 以外の名前空間に対して using namespace を使っている）:

    ha_mroonga.cpp:
      using namespace zmq;

  include ガード

2重 include を防ぐためのマクロは、ヘッダーファイルの名前をすべて大文字にし、単語の区切りをアンダースコアにしたものにし、最後にアンダースコアをつけて HEADER_FILE_NAME_HPP_ という名前にする。

よい例:

    mrn_db_path.hpp:
      #ifndef MRN_DB_PATH_HPP_
      #define MRN_DB_PATH_HPP_
      ...
      #endif // MRN_DB_PATH_HPP_

悪い例（小文字になっている）:

    mrn_db_path.hpp:
      #ifndef mrn_db_path_hpp_
      #define mrn_db_path_hpp_
      ...
      #endif // mrn_db_path_hpp_

悪い例（アンダースコアが最後ではなく先頭についている）:

    mrn_db_path.hpp:
      #ifndef _MRN_DB_PATH_HPP
      #define _MRN_DB_PATH_HPP
      ...
      #endif // _MRN_DB_PATH_HPP

  代入

= の前後に1つスペースを入れる。

よい例:

    int i = 0;

悪い例（スペースが入っていない）:

    for (i=0; i<10; ++i) {...}

悪い例（2つスペースが入っている）:

    int i  = 0;

  文字列

文字列はポインタと長さで表現する。 \0 での終端を仮定しない。

よい例（本当はもっとすっきりした例がよいけど。。。）:

    char *raw_data = "table_name column_name column_value"
    char *column_name;
    size_t column_name_size;
    column_name = raw_data + strlen("table_name ");
    column_name_size = strlen("column_name");

悪い例（無理やり \0 終端にしている）:

    char *raw_data = "table_name column_name column_value"
    char *column_name;
    column_name = strndup(raw_data + strlen("table_name "), strlen("column_name"));

ただし、ファイル名など \0 が前提であるものに関しては \0 終端を仮定してよい。

よい例:

    char *database_path = "db/test.mrn";

悪い例（ \0 終端を仮定せず、長さも管理している）:

    char *database_path = "db/test.mrn";
    size_t database_path_size = strlen("db/test.mrn");

std::string は内部でメモリ確保などの処理が発生するので多用しない。

よい例:

    char database_path[MAX_PATH];

悪い例（最大サイズがわかっているのに std::string を使っている）:

    std::string database_path;

  バッファ

TODO: ちゃんと考える。

何度も繰り返し使う領域（バッファ）が必要な場合は GRN_BULK を使う。例えば、カラムの値を取得する領域などである。

  命名規則

  クラス名

クラスの名前は UpperCamelCase とする。

よい例:

    class MyClass
    {
    }

悪い例（ snail_case である）:

    class my_class
    {
    }

ただし、 ha_mroonga などMySQLとのインターフェイスとなるクラスでかつ他の類似のモジュールに命名規則がある場合はそれに従う。

よい例:

    class ha_mroonga: public handler
    {
    }

悪い例（ UpperCamelCase になっている）:

    class HaMroonga: public handler
    {
    }

  メンバー変数名

メンバー変数名は snail_case とし、末尾にアンダースコア（ _ ）を付ける。

よい例:

    class MyClass
    {
      char *my_name_;
    }

悪い例（ UpperCamelCase である）:

    class MyClass
    {
      char *MyName_;
    }

悪い例（末尾にアンダースコアがない）:

    class MyClass
    {
      char *my_name;
    }

  読み込み用アクセサ名

メンバー変数の値を読み込むメソッドの名前はメンバー変数名の末尾のアンダースコアを除いたものにする。

よい例:

    class MyClass
    {
      char *my_name_;
      const char *my_name() {return my_name_;};
    }

悪い例（末尾にアンダースコアが残っている）:

    class MyClass
    {
      char *my_name_;
      const char *my_name_() {return my_name_;};
    }

悪い例（先頭に get_ を付けている）:

    class MyClass
    {
      char *my_name_;
      const char *_my_name() {return my_name_;};
    }

  書き込み用アクセサ名

メンバー変数の値を設定するメソッドの名前は、メンバー変数名の末尾のアンダースコアを除き、先頭に set_ を加えたものにする。

よい例:

    class MyClass
    {
      unsigned int age_;
      void set_age(unsigned int age)
      {
        age_ = age;
      };
    }

悪い例（末尾にアンダースコアが残っている）:

    class MyClass
    {
      unsigned int age_;
      void set_age_(unsigned int age)
      {
        age_ = age;
      };
    }

悪い例（先頭に set_ ではなく update_ を付けている）:

    class MyClass
    {
      unsigned int age_;
      void update_age(unsigned int age)
      {
        age_ = age;
      };
    }

  コピーコンストラクター

基本的にコピーコンストラクターの使用を禁止する。よほどのことがなければ使用しないこと。

コピーコンストラクターは暗黙的に無駄なコピーが発生する可能性があるためパフォーマンス上の問題がある。コピーではなくポインターやリファレンスを用いること。

また、デフォルトのコピーコンストラクター実装はメンバー変数のポインターの値をそのままコピーするため、デコンストラクターで二重に解放してしまう危険性がある。そのため、明示的にコピーコンストラクターを定義しない場合は無効にする。

よい例:

    class MyClass
    {
    private:
      MyClass(const MyClass &);
    }

悪い例（コピーコンストラクターを禁止していない）:

    class MyClass
    {
    }

悪い例（カスタムコピーコンストラクターを使っている）:

    class MyClass
    {
      unsigned int age_;
      MyClass(const MyClass &object)
      {
        age_ = object.age_;
      }
    }

  クラスの代入

基本的に定義したクラスの代入を禁止する。よほどのことがなければ使用しないこと。

代入演算子は暗黙的に無駄なコピーが発生する可能性があるためパフォーマンス上の問題がある。コピーではなくポインターやリファレンスを用いること。

また、デフォルトの代入演算子の実装はメンバー変数のポインターの値をそのままコピーするため、デコンストラクターで二重に解放してしまう危険性がある。そのため、明示的に代入演算子を定義しない場合は無効にする。

よい例:

    class MyClass
    {
    private:
      MyClass &operator=(const MyClass &);
    }

悪い例（代入を禁止していない）:

    class MyClass
    {
    }

悪い例（代入を使っている）:

    class MyClass
    {
      unsigned int age_;
      MyClass &operator=(const MyClass &object)
      {
        age_ = object.age_;
        return *this;
      }
    }

  引数

  voidを省略

引数がない場合は void を省略する。

よい例:

    class MyClass
    {
      unsigned int age_;
      unsigned int age()
      {
         return age_;
      };
    }

悪い例（ void を省略していない）:

    class MyClass
    {
      unsigned int age_;
      unsigned int age(void)
      {
         return age_;
      };
    }

  入力用引数にはconstを付ける

入力のみに用いる引数には const を付ける。これは、入力のみに用いる引数である事を明示するためと、間違って引数を変更してしまわないためである。

よい例:

    class Table
    {
      void insert(unsigned int id, const char *column_name, const char *value)
      {
         Record *record = records[i];
         Column *column = columns[column_name];
         column.set_value(value);
      }
    }

悪い例（入力のみに用いているのに const が付いていない）:

    class Table
    {
      void insert(unsigned int id, char *column_name, char *value)
      {
         Record *record = records[i];
         Column *column = columns[column_name];
         column.set_value(value);
      }
    }

  定数

フラグやサイズなどを示す定数には const オブジェクトを用いる。これはデバッガー上でプログラムを走らせているときに名前で値を参照できるようにするためである。

よい例:

    const char *MRN_LOG_FILE_PATH = "groonga.log";

悪い例（ #define を用いている）:

    #define MRN_LOG_FILE_PATH "groonga.log"

  真偽値

  bool型を用いる

真偽値には bool 型を用いる。

よい例:

    bool is_searching;

悪い例（ int 型を用いている）:

    int is_searching;

  真偽値のリテラルには true または false を用いる

真偽値の値として true と false はより完結で説明的だからである。

よい例:

    bool is_searching = true;

悪い例（ 0 以外の値を真の値として用いている）:

    bool is_searching = 1;

  条件式

  真偽値は比較しない

真偽値の値は boolean_value == true などとせず、 boolean_value として条件式に使用する。すでに真偽値の値を真偽値のリテラルと比較することは重複したコードだからである。

よい例:

    boolean is_searching = true;
    if (!is_searching) { ... }

悪い例（真偽値のリテラルと比較している）:

    boolean is_searching = true;
    if (is_searching == false) { ... }

  NULL と比較しない

NULL かどうかを条件式に使う場合は value == NULL ではなく !value というように書く。多くの言語で NULL に相当する値（たとえばLispの nil ）は偽を表すため、明示的に NULL と比較しなくても意図は伝わるからである。

よい例:

    char *name = NULL;
    if (!name) { ... }

悪い例（ NULL と比較している）:

    char *name = NULL;
    if (name == NULL) { ... }

  数値は比較する

CやC++では 0 は偽、 0 以外は真の値となるが、条件式に数値を使う場合は strcmp(...) == 0 などというように明示的に比較する。

C++では真偽値に bool を使うためこのような状況は発生しないが、C言語由来のAPIでは int で真偽値を表現している場合が多い。しかし、 int だけでは真偽値として使っているか本当に数値として使っているかがわかりにくいため、 int のときはすべて数値として扱う。

よい例:

    if (memcmp(value1, value2, value_size) == 0) {
      printf("same value!\n");
    }

悪い例（ 0 を偽の値として扱っている）:

    if (!memcmp(value1, value2, value_size)) {
      printf("same value!\n");
    }

  初期化

  メンバー変数の初期化には初期化リストを用いる

無駄な処理を省くためにコンストラクターでのメンバー変数の初期化には初期化リストを用いる。初期化リストを用いないとコンストラクターの処理とコピーコンストラクター・代入処理が行われたりなど非効率である。（後述）

よい例:

    class Table
    {
      Table(const char *name);
      std::string name_;
    }
    Table::Table(const char *name) :
      name_(name)
    {
    }

悪い例（ std::string(name) のところでコンストラクターが動き、 name_ = ... のところで代入演算子が動いて2回初期化している）:

    class Table
    {
      Table(const char *name);
      std::string name_;
    }
    Table::Table(const char *name)
    {
      name_ = std::string(name);
    }

  変数宣言と同時に初期化する

変数を宣言したときに同時に初期化する。宣言時に初期化せずに代入して初期化すると、無駄な処理が発生する可能性があるため非効率である。（後述）

よい例:

    std::string name("users");

悪い例（ std::string() のところでコンストラクターが動き、 name = ... のところで代入演算子が動いて2回初期化している）:

    std::string name;
    name = std::string("users");

  インクリメント・デクリメント

  前置形式を用いる

後置形式ではオブジェクトのコピーをしなければいけないため非効率である。そのため、できるだけ前置形式を用いる。

よい例（ int だと効率は変わらないので本当はあんまりよい例ではない）:

    for (int i = 0; i < 10; ++i) {
    }

悪い例（後置形式を用いている）:

    for (int i = 0; i < 10; ++i) {
    }

  キャスト

  C++のスタイルを用いる

Cスタイルのキャストはなんでもキャストできてしまうため、意図しないキャストにも気付かない可能性がある。例えば、単に const を外したいだけなのに、間違って違う型に変換していても気付けない。C++のキャストでは const を外したいときは const_cast を使用し、型を変換するときは static_cast を指定する。こうすれば、 static_cast で間違って const を外してしまっている場合も気付ける。 reinterpret_cast はどうしても必要なときのみ注意して使う。

よい例（ const_cast を使っている）:

    uchar *to_key;
    const ucahr *from_key;
    KEY *key_info;
    uint key_length;
    key_copy(to_key, const_cast<uchar *>from_key, key_info, key_length);

よい例（ static_cast を使っている）:

    int n_hits = 1;
    int n_documents = 10;
    float hit_ratio = (float)(n_hits) / n_documents;

よい例（ static_cast では無理なので reinterpret_cast を使っている）:

    THD *thread = current_thd;
    my_hash_delete(&mrn_allocated_thds, reinterpret_cast<uchar *>(thread));

悪い例（Cスタイルのキャストを使っている）:

    int n_hits = 1;
    int n_documents = 10;
    float hit_ratio = (float)(n_hits) / n_documents;

悪い例（ static_cast で十分なのに reinterpret_cast を使っている）:

    void *value = get_value(key);
    char *name;
    name = reinterpret_cast<char *>(value);

  変数宣言

  ポインタ型を示す * とリファレンス型を示す & は変数名に寄せる

Cと同様にポインタ型を示す * は型名ではなく変数名に寄せる。これは、以下のように複数の変数を一度に宣言したときに一貫性がなくなるためである。2つめ以降の変数は近くに型名がないため * を寄せる場所がない。

例:

    char* key, *value;

同様に、リファレンス型を示す & も変数名に寄せる。

なお、 * や & と型名の間にはスペースを入れない。

よい例:

    char *key;

よい例:

    bool is_exist(const std::string &file_name);

悪い例（型名に寄せている）:

    char* key;

  その他

•   ここに書いていないものについては特にスタイルを定めないが、 プロジェクト内で常に一貫性のあるスタイルを使用すること。 同じような事をするときは同じような書き方にすること。 複数の書き方で同じようなことを実現している場合は1つの方法に合わせること。

    以下、具体例が必要。

•   ビルド時にできることを実行時に延ばさない(静的チェックを活用)

•   なるべく局所的に変数を定義し、同時に初期化する

•   長い関数や深いブロックのネストを避ける

•   2つに分けた方がよさそう。あと目安があるといいかも。 100行以上は長いよねーとか3段以上はデンジャーとか。

•   必要以上にオブジェクトを複製しない

•   暗黙の型変換はなるべく避ける

•   assertを積極的に使う

  メモ

•   動的テンプレートを使わない。（要追加情報。implicit template instantiationのことであれば、これはふつうに使わ れているものなので特に禁止しないんでいいんじゃない説？）

•   typeidを使わない。

•   例外はMySQLで問題がないようであればOK。Mroongaから外の世 界（MySQLの世界）にはださないようにする。

  How we are developing

The Mroonga project uses Redmine for Ticket Driven Development.

The location of our Redmine site is the following.

<http://redmine.groonga.org/projects/mroonga>

Developments are done per ticket.

We would like to ask developers to register on the site above.

  Roadmap

We develop based on the roadmap, and you can see our roadmap in the following page.

<http://redmine.groonga.org/projects/mroonga/roadmap>

We draw up our roadmap by discussing in off-line or on-line meetings.

The roadmap specifies the list of adding features in each version.

  Source code management

We manage the source code using Git on Github.

<http://github.com/mroonga/mroonga>

If you want to contribute, please make clone in read/write mode, and then commit and push.

If you still do not have your account on github.com, please register.

  Development process

We develop Mroonga using the following procedures.

1.  Create a new ticket and add a description

2.  Discuss in the development meeting and update roadmap

3.  Assign the ticket, and the assignee starts development

4.  Implement features or fix bugs, then push to the repository if needed

5.  Add and run test codes, then push to the repository

6.  Review the implementation and tests, and go back to 4. if needed

7.  Add or update documents, then push to the repository

8.  Close the ticket

    We welcome your ideas about new features or changes of specifications. Please create a ticket first and describe your idea there.

    For adding and running test codes, please refer the description below.

    We want to ask the assignee to handle whole the process, like design, implementation, test and documentations.

    Then you ask other developers to review (especially in case of adding new features), and when it passes you add or update documents and the ticket can be closed.

    For adding and updating documents, please refer the description below.

  Development environment

Currently we are assuming the following development environments.

•   Linux x86_64

•   glibc

•   MySQL <http://www.mysql.com/>

•   Groonga <https://groonga.org/>

•   Cutter <http://cutter.sourceforge.net/> (for C/C++ unit tests)

•   Sphinx <https://www.sphinx-doc.org/en/master/> (for documents)

  Contents of the source tree

There are just a few source files for now, and we would like to keep it simple as possible.

ha_mroonga.hpp
    The header file of Mroonga.

ha_mroonga.cpp
    The implementation of Mroonga.

mrnsys.hpp
    The header file of utility functions

mrnsys.cpp
    The implementation of utility functions

mysql-test/
    The directory for tests by SQL

    SQL test scripts are included in each test case 't/' directory, that are also the definition of currently available SQL statements.

    The expected results of SQL tests are included in each test case 'r/' directory , that are also the definition of the current specification of supported SQL.

test/unit/
    The directory for per-function unit tests by C/C++

doc/source/
    English documents in Sphinx format

    Since we are still in the early stage of the development of Mroonga, we will not make the documents of the specification of SQL queries for now.

    Alternatively we consider our SQL tests and its expected results as the list of features and the definition of their specifications.

  Adding and running tests

We use two kinds of regression tests to manage the quality of Mroonga.

SQL tests
    When you add features or fix bugs that can be confirmed by SQL queries, please always add SQL tests. You might think that performance improvements cannot be confirmed with SQL, but some can be still well tested by using status variables or information_schema plugin etc.

C/C++ unit tests
    They are function level regression tests using Cutter. When you add features that cannot differ SQL queries' results, like utility functions, please add tests here.

    Before pushing to the repository, please always run the regression tests and confirm that you don't introduce any degradation.

    You can invoke these two kinds of tests by "make check".

    SQL tests are implemented as "sub test suite" for "mysql-test" in MySQL's regression tests. For the detail about how to add test cases or how to modify expected result files, please refer the following MySQL document.

    <http://dev.mysql.com/doc/mysqltest/2.0/en/index.html>

    For the detail about C/C++ unit tests, please refer the following Cutter document.

    <http://cutter.sourceforge.net/>

  Adding and updating documents

See How to contribute in documentation topics <> about adding and updating documents.

•   Index <>

•   Module Index <>

•   Search Page <>

Author
Mroonga Project

Copyright
2009-2026, Mroonga Project