com.yahoo.bullet.querying.partitioning

Class SimpleEqualityPartitioner

java.lang.Object

com.yahoo.bullet.querying.partitioning.SimpleEqualityPartitioner

All Implemented Interfaces:

Partitioner

public class SimpleEqualityPartitioner
extends Object
implements Partitioner

This partitioner uses a list of fields to partition. If fields A and B are used to partition, this partitioner tries to make sure that queries with equality filters on A and/or B are partitioned appropriately and makes sure that records with values for A and/or B end up seeing a subset of queries that have ANDed, equality filters for those values. This simple partitioner is only to be used for partitioning queries with ANDed equality filters with one value for each filter. The latter criteria can be relaxed in the future but for now, it is not supported. It will default partition the query if: 1) ORs or NOTs are detected in the filter 2) If a field being equality filtered on is present with multiple values in filter 3) If the query has no filters Ex: A == foo AND B.c == bar AND D == null, using the fields [A, B.c, D] will make sure that records will values of foo, bar and null for those fields, will be seen only by those queries that are have those filters (or subsets of it), including queries with no filters on these fields (and queries with no filters at all). The getKeys(Query) returns a set of size 1. This means the queries need not be duplicated after partitioning. However, getKeys(BulletRecord) will return a set of keys representing the queries that this record needs to presented to. The size of this can be up to 2^(# of fields), where each of the keys is the join of all combinations of null and the actual value in the record for each field in field order with the delimiter. Ex: If fields are [A, B.c, D] and a record has these values: [A: foo, B.c: bar, D: baz, ...], the keys will be the the concatenation of the following items in each tuple using the configured delimiter (not necessarily in this order): [foo, bar, baz] [foo, null, baz] [foo, bar, null] [foo, null, null] [null, bar, baz] [null, bar, null] [null, null, baz] [null, null, null] Using these keys and presenting the record to all the queries with the same key will ensure that the record is seen by exactly only the queries that need to see it.

Field Summary

Fields

Modifier and Type	Field and Description
static char	DISAMBIGUATOR

Constructor Summary

Constructors

Constructor and Description
SimpleEqualityPartitioner(BulletConfig config) Constructor that takes a BulletConfig instance with definitions for the various settings this needs.

Method Summary

Modifier and Type	Method and Description
Set<String>	getKeys(com.yahoo.bullet.record.BulletRecord record) Returns the partitioning keys for this BulletRecord instance.
Set<String>	getKeys(Query query) Returns the partitioning keys for this Query instance.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DISAMBIGUATOR

public static final char DISAMBIGUATOR

See Also:

Constant Field Values

Constructor Detail

SimpleEqualityPartitioner

public SimpleEqualityPartitioner(BulletConfig config)

Constructor that takes a BulletConfig instance with definitions for the various settings this needs. Delimiter: BulletConfig.EQUALITY_PARTITIONER_DELIMITER and Fields to partition on: BulletConfig.EQUALITY_PARTITIONER_FIELDS

Parameters:

config - The non-null config containing settings for this class.

Method Detail

getKeys

public Set<String> getKeys(Query query)

Returns the partitioning keys for this Query instance. This partitioner ensures that queries are not stored in duplicate by returning only key for a query (the list that is returned is of size 1).

Specified by:

getKeys in interface Partitioner

Parameters:

query - The query to partition for.

Returns:

A non-null Set of Strings representing the keys for this query.

getKeys

public Set<String> getKeys(com.yahoo.bullet.record.BulletRecord record)

Description copied from interface: Partitioner

Returns the partitioning keys for this BulletRecord instance.

Specified by:

getKeys in interface Partitioner

Parameters:

record - The record to partition.

Returns:

A non-null Set of Strings representing the keys for this record.