Redis implementation Bloom filter (below)

Redis4.0 integrates the Bloom filter through a modular form. After that, the Bloom filter can be operated by the following command, the path https://redis.io/commands/?group=bf

So how do we use the Java code to operate the Bloom filter?

Redisbloom's official website proposes three ways to directly operate the Redis Bloom filter module. Refer to the relevant information refer to https://github.com/redisblow/redisbloom, where Jrebloom has discussed for the time being.

Note: When remote connection, the protection mode needs to be closed in the redis.conf configuration fileprotected-mode no。

Jedis supports Buron filter

Jedis is a simple and easy -to -use client designed for Redis. At the same time, the latest version also supports modular services such as Bloom filter

Dependence

<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>4.2.0</version>
</dependency>

Test code

public class JedisDemo {
    public static void main(String[] args) throws URISyntaxException {
         // establish connection 
        BloomFilterCommands bloomFilterCommands = new JedisPooled(new URI("redis://47.99.147.139:6379"),1000000);

 // Construct a Bloom filter parameter
        BFReserveParams bfReserveParams = new BFReserveParams();
        bfReserveParams.expansion(2);

 // Create a filter
        String test = bloomFilterCommands.bfReserve("test", 0.1, 100, bfReserveParams);

 // Add elements to the filter
        bloomFilterCommands.bfAdd("test",String.valueOf(1));

 // Determine whether the element exists
        boolean isExist = bloomFilterCommands.bfExists("test", "1");

        System.out.println(isExist);
    }
}

The remaining functional APIs can check https://www.javadoc.io/doc/redis.clients/jedis/Latest/index.html.

Redis-Modules supports Bloom filter

Redis-Modules is a Java-based client that depends on Redisson, so it also needs to introduce redisson dependencies.

Dependence

<!-Introduce redisson dependencies->
<dependency>
   <groupId>org.redisson</groupId>
   <artifactId>redisson</artifactId>
   <version>3.17.1</version>
</dependency>  

 <!-Release here is the introduction of all supporting Module->
<dependency>
    <groupId>io.github.dengliming.redismodule</groupId>
    <artifactId>all</artifactId>
    <version>2.0.0</version>
</dependency>
 <!-You can also introduce a single Module: such as the Bloom filter, just choose one of them->
<dependency>
    <groupId>io.github.dengliming.redismodule</groupId>
    <artifactId>redisbloom</artifactId>
    <version>2.0.0</version>
</dependency>

Test code

public static void main(String[] args) {
 // Redisson creates the same way of connecting, you need to introduce redisson dependencies
    Config config = new Config();
    config.useSingleServer().setAddress("redis://127.0.0.1:6379");
    RedisBloomClient redisBloomClient = new RedisBloomClient(config);

 // Object to get the specified name
    BloomFilter bloomFilter = redisBloomClient.getRBloomFilter("bf");
 // Specify the error rate and capacity of the Bloom filter
    bloomFilter.create(0.1d, 100);
 // Add elements to the Bloom filter
    bloomFilter.madd(new String[] {"a", "b", "c"});
 // Black judgment element is similar to BF.Mexists
    List<Boolean> booleans = bloomFilter.existsMulti("a", "b", "d");
    System.out.println(booleans);

 // Close the client
    redisBloomClient.shutdown();
}

The corresponding online link of the redis Bronon filter command and the Java code can refer to the official online link https://github.com/dengliming/redis-modules-java/master/redisblow

The above-mentioned Jedis and Redis-Modules support the Bloom filter, which is stored in the form of a Buron filter in the Redis

But in addition to this form, there are other methods to implement the Bloom filter. For example, Redisson does not need to use Redis-Bloom integrated Redis-Bloom to achieve the Bloom filter.

Redisson supports Bloom filter

Dependence

<dependency>
   <groupId>org.redisson</groupId>
   <artifactId>redisson</artifactId>
   <version>3.17.1</version>
</dependency>  

Test connection code

public static void main(String[] args) {
    // 1. Create config object
    Config config = new Config();
 /** Select the instantiation method according to the redis instance mode
 *Cluster mode: UseClusterservers
 *Single example mode: useSingLeserver
 * SENTINEL mode: UseSEntinelServers
 *Main Mode: UseMasterSLAVESERVERS
       */
    config.useSingleServer().setAddress("redis://47.99.147.139:6379");

    // 2. Create Redisson instance
    RedissonClient redisson = Redisson.create(config);

    RBloomFilter bloomFilter = redisson.getBloomFilter("sample");
 // Initialize Bloom Filter with must use Tryinit initialization before use
 // ExpectedInsertions = 55000000 expected value
 // falseProbability = 0.03 error rate
    bloomFilter.tryInit(10, 0.03);
    bloomFilter.add("1001");

    System.out.println(bloomFilter.contains("1002"));

    redisson.shutdown();
}

After executing the above code, we can view the data stored in Redis

### Redisson execute BloomFilter.tryinit, and succeed,
 ### Do not add elements to the Bloom filter, so the redis is only one value {**}: config
127.0.0.1:6379> keys *
1) "{sample}:config"
 ### View the type of deposit value
127.0.0.1:6379> type "{sample}:config"
hash
127.0.0.1:6379> HKEYS "{sample}:config"
 1) "SIZE" ### Binary BIT array length
 2) "Hashitations" ### Hao Function
 3) "ExpectedInsertions" ### Capacity
 4) "FalseProbability" ### error rate
127.0.0.1:6379> hget "{sample}:config" "size"
"72"
127.0.0.1:6379> hget "{sample}:config" "hashIterations"
"5"
127.0.0.1:6379> hget "{sample}:config" "expectedInsertions"
"10"
127.0.0.1:6379> hget "{sample}:config" "falseProbability"
"0.03"
 #### Storing the real value
127.0.0.1:6379> type sample
string

Redisson generates two key values ​​{SAMPLE}: Config and Sample. Among them, {SAMPLE}: Config is a hash type, which is used to store the Bloom filter configuration of the user configuration. Essence

So is there any way to achieve the Bloom filter except relying on Redis? Of course, Google can open source GUAVA tools to achieve Buron filter.

GUAVA realizes Bloom filter

GUAVA is a Google open source tool class, which can directly achieve the Bloom filter, without requiring repeated wheels.

Dependence

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>31.1-jre</version>
</dependency>

Test code

public static void main(String[] args) {
    int num = 100000;
// Create the object of the Bloom Filter, the misjudgment rate is 0.03 by default, you can specify
 // Num means the capacity of the Bloom filter
    BloomFilter bloomFilter = BloomFilter.create(Funnels.integerFunnel(),num,0.01);

 // Add data to the Bloom filter
    for (int i = 0; i <num ; i++) {
        bloomFilter.put(i);
    }

 // Judging the misjudgment of the value just added
    for (int i = 0; i <num ; i++) {
 // MightContain determines whether the Bloom filter contains objects
        if (!bloomFilter.mightContain(i)){
 System.out.println ("Misunderstanding ~, Value:"+I+"== is");
        }
    }

    int count = 0;
 // Determine whether the value in the Brunt filter is misjudged
    for (int i = num; i <num+num ; i++) {
        if (bloomFilter.mightContain(i)){
            count += 1;
        }
    }
 System.out.println ("There is no element misjudgment in the Bloom filter:"+Count);
}

Results show that there is a misjudgment, which is in line with the pre -set error range

For other commands, you can refer to the guava official website API document https://guava.dev/releases/snapshot-jre/api/docs/google/common/blowfilter.html.html

Buron filter configuration reference

We can enter the Bloom Filter size and acceptable error rates through the open source tools on GitHub, calculate the best configuration of the Bloom filter, calculate the tool refer to https://krisives.github.io/bloom-calculator/

The error rate is related to the hash function and the memory capacity. When the error rate is lower, the more the number of hash functions you need, the larger the memory capacity, the larger the memory capacity. This is a choice in accuracy and performance.

The application scenario of the Bloom Filter

Although the Bloom filter has the disadvantages of misjudging and deleting difficulties, it can quickly determine whether the characteristics of the object exist are still widely used, commonly used in

• Prevent redis cache penetration.

• Crazy filtering, the arrested URL no longer captures it.

• Spam filtering, etc.