Packages

trait DataProducer extends AnyRef

Trait that can be implemented to produce BulletRecord data as input to Bullet running on Spark Streaming.

This trait is used to plugin users' source of data to Bullet. The data can be from anywhere. For example, Kafka, HDFS, H3, Kinesis etc. This generally involves hooking in a Receiver. Users can also do any transformations on the data before emitting it to Bullet in their own implementations by using the provided Spark Streaming Context. This will be the same context used to wire in the rest of the Bullet DAG.

Linear Supertypes
AnyRef, Any
Known Subclasses
DSLDataProducer
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DataProducer
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def getBulletRecordStream(ssc: StreamingContext, config: BulletSparkConfig): DStream[BulletRecord[_ <: Serializable]]

    Get Bullet record stream from users' source of data.

    Get Bullet record stream from users' source of data.

    In this method, any transformations to the users' data can be done.

    ssc

    The StreamingContext that can be used to define an arbitrary DAG to compute the BulletRecord stream.

    config

    The com.yahoo.bullet.spark.utils.BulletSparkConfig containing all the settings.

    returns

    The BulletRecord stream, which will be used as the input to Bullet.