org.apache.predictionio

controller

package controller

Provides building blocks for writing a complete prediction engine consisting of DataSource, Preparator, Algorithm, Serving, and Evaluation.

Start Building an Engine

The starting point of a prediction engine is the Engine class.

The DASE Paradigm

The building blocks together form the DASE paradigm. Learn more about DASE here.

Types of Building Blocks

Depending on the problem you are solving, you would need to pick appropriate flavors of building blocks.

Engines

There are 3 typical engine configurations:

  1. PDataSource, PPreparator, P2LAlgorithm, LServing 2. PDataSource, PPreparator, PAlgorithm, LServing 3. LDataSource, LPreparator, LAlgorithm, LServing

In both configurations 1 and 2, data is sourced and prepared in a parallelized fashion, with data type as RDD.

The difference between configurations 1 and 2 come at the algorithm stage. In configuration 1, the algorithm operates on potentially large data as RDDs in the Spark cluster, and eventually outputs a model that is small enough to fit in a single machine.

On the other hand, configuration 2 outputs a model that is potentially too large to fit in a single machine, and must reside in the Spark cluster as RDD(s).

With configuration 1 (P2LAlgorithm), PredictionIO will automatically try to persist the model to local disk or HDFS if the model is serializable.

With configuration 2 (PAlgorithm), PredictionIO will not automatically try to persist the model, unless the model implements the PersistentModel trait.

In special circumstances where both the data and the model are small, configuration 3 may be used. Beware that RDDs cannot be used with configuration 3.

Data Source

PDataSource is probably the most used data source base class with the ability to process RDD-based data. LDataSource cannot handle RDD-based data. Use only when you have a special requirement.

Preparator

With PDataSource, you must pick PPreparator. The same applies to LDataSource and LPreparator.

Algorithm

The workhorse of the engine comes in 3 different flavors.

P2LAlgorithm

Produces a model that is small enough to fit in a single machine from PDataSource and PPreparator. The model cannot contain any RDD. If the produced model is serializable, PredictionIO will try to automatically persist it. In addition, P2LAlgorithm.batchPredict is already implemented for Evaluation purpose.

PAlgorithm

Produces a model that could contain RDDs from PDataSource and PPreparator. PredictionIO will not try to persist it automatically unless the model implements PersistentModel. PAlgorithm.batchPredict must be implemented for Evaluation.

LAlgorithm

Produces a model that is small enough to fit in a single machine from LDataSource and LPreparator. The model cannot contain any RDD. If the produced model is serializable, PredictionIO will try to automatically persist it. In addition, LAlgorithm.batchPredict is already implemented for Evaluation purpose.

Serving

The serving component comes with only 1 flavor--LServing. At the serving stage, it is assumed that the result being served is already at a human- consumable size.

Model Persistence

PredictionIO tries its best to persist trained models automatically. Please refer to LAlgorithm.makePersistentModel, P2LAlgorithm.makePersistentModel, and PAlgorithm.makePersistentModel for descriptions on different strategies.

Linear Supertypes
AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By inheritance
Inherited
  1. controller
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. abstract class AverageMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Double]

    Returns the global average of the score returned by the calculate method.

  2. trait CustomQuerySerializer extends BaseQuerySerializer

    If your query class cannot be automatically serialized/deserialized to/from JSON, implement a trait by extending this trait, and overriding the querySerializer member with your custom JSON4S serializer.

  3. trait Deployment extends EngineFactory

    Defines a deployment that contains an Engine

  4. type EmptyActualResult = SerializableClass

    Empty actual result.

  5. type EmptyAlgorithmParams = EmptyParams

    Empty algorithm parameters.

  6. type EmptyDataParams = EmptyParams

    Empty data parameters.

  7. type EmptyDataSourceParams = EmptyParams

    Empty data source parameters.

  8. type EmptyEvaluationInfo = SerializableClass

    Empty evaluation info.

  9. type EmptyMetricsParams = EmptyParams

    Empty metrics parameters.

  10. type EmptyModel = SerializableClass

    Empty model.

  11. case class EmptyParams() extends Params with Product with Serializable

    A concrete implementation of Params representing empty parameters.

  12. type EmptyPreparatorParams = EmptyParams

    Empty preparator parameters.

  13. type EmptyPreparedData = SerializableClass

    Empty prepared data.

  14. type EmptyServingParams = EmptyParams

    Empty serving parameters.

  15. type EmptyTrainingData = SerializableClass

    Empty training data.

  16. class Engine[TD, EI, PD, Q, P, A] extends BaseEngine[EI, Q, P, A]

    This class chains up the entire data process.

  17. abstract class EngineFactory extends AnyRef

    If you intend to let PredictionIO create workflow and deploy serving automatically, you will need to implement an object that extends this class and return an Engine.

  18. class EngineParams extends Serializable

    This class serves as a logical grouping of all required engine's parameters.

  19. trait EngineParamsGenerator extends AnyRef

    Defines an engine parameters generator.

  20. trait Evaluation extends EngineFactory with Deployment

    Defines an evaluation that contains an engine and a metric.

  21. class FastEvalEngine[TD, EI, PD, Q, P, A] extends Engine[TD, EI, PD, Q, P, A]

    :: Experimental :: FastEvalEngine is a subclass of Engine that exploits the immutability of controllers to optimize the evaluation process

  22. class FastEvalEngineWorkflow[TD, EI, PD, Q, P, A] extends Serializable

    :: Experimental :: Workflow based on FastEvalEngine

  23. class IdentityPreparator[TD] extends BasePreparator[TD, TD]

    A helper concrete implementation of org.apache.predictionio.core.BasePreparator that passes training data through without any special preparation.

  24. abstract class LAlgorithm[PD, M, Q, P] extends BaseAlgorithm[RDD[PD], RDD[M], Q, P]

    Base class of a local algorithm.

  25. class LAverageServing[Q] extends LServing[Q, Double]

    A concrete implementation of LServing returning the average of all algorithms' predictions, where their classes are expected to be all Double.

  26. abstract class LDataSource[TD, EI, Q, A] extends BaseDataSource[RDD[TD], EI, Q, A]

    Base class of a local data source.

  27. class LFirstServing[Q, P] extends LServing[Q, P]

    A concrete implementation of LServing returning the first algorithm's prediction result directly without any modification.

  28. abstract class LPreparator[TD, PD] extends BasePreparator[RDD[TD], RDD[PD]]

    Base class of a local preparator.

  29. abstract class LServing[Q, P] extends BaseServing[Q, P]

    Base class of serving.

  30. trait LocalFileSystemPersistentModel[AP <: Params] extends PersistentModel[AP]

    This trait is a convenience helper for persisting your model to the local filesystem.

  31. trait LocalFileSystemPersistentModelLoader[AP <: Params, M] extends PersistentModelLoader[AP, M]

    Implement an object that extends this trait for PredictionIO to support loading a persisted model from local filesystem during serving deployment.

  32. abstract class Metric[EI, Q, P, A, R] extends Serializable

    Base class of a Metric.

  33. class MetricEvaluator[EI, Q, P, A, R] extends BaseEvaluator[EI, Q, P, A, MetricEvaluatorResult[R]]

    :: DeveloperApi :: Do no use this directly.

  34. case class MetricEvaluatorResult[R](bestScore: MetricScores[R], bestEngineParams: EngineParams, bestIdx: Int, metricHeader: String, otherMetricHeaders: Seq[String], engineParamsScores: Seq[(EngineParams, MetricScores[R])], outputPath: Option[String]) extends BaseEvaluatorResult with Product with Serializable

    Contains all results of a MetricEvaluator

  35. case class MetricScores[R](score: R, otherScores: Seq[Any]) extends Product with Serializable

    Case class storing a primary score, and other scores

  36. abstract class OptionAverageMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsOptionMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Option[Double]]

    Returns the global average of the non-None score returned by the calculate method.

  37. abstract class OptionStdevMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsOptionMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Option[Double]]

    Returns the global standard deviation of the non-None score returned by the calculate method

  38. abstract class P2LAlgorithm[PD, M, Q, P] extends BaseAlgorithm[PD, M, Q, P]

    Base class of a parallel-to-local algorithm.

  39. abstract class PAlgorithm[PD, M, Q, P] extends BaseAlgorithm[PD, M, Q, P]

    Base class of a parallel algorithm.

  40. abstract class PDataSource[TD, EI, Q, A] extends BaseDataSource[TD, EI, Q, A]

    Base class of a parallel data source.

  41. abstract class PPreparator[TD, PD] extends BasePreparator[TD, PD]

    Base class of a parallel preparator.

  42. trait Params extends Serializable

    Base trait for all kinds of parameters that will be passed to constructors of different controller classes.

  43. trait PersistentModel[AP <: Params] extends AnyRef

    Mix in and implement this trait if your model cannot be persisted by PredictionIO automatically.

  44. trait PersistentModelLoader[AP <: Params, M] extends AnyRef

    Implement an object that extends this trait for PredictionIO to support loading a persisted model during serving deployment.

  45. trait QPAMetric[Q, P, A, R] extends AnyRef

    Trait for metric which returns a score based on Query, PredictedResult, and ActualResult

  46. trait SanityCheck extends AnyRef

    Extends a data class with this trait if you want PredictionIO to automatically perform sanity check on your data classes during training.

  47. class SerializableClass extends Serializable

    Base class of several helper types that represent emptiness

  48. class SimpleEngine[TD, EI, Q, P, A] extends Engine[TD, EI, TD, Q, P, A]

    SimpleEngine has only one algorithm, and uses default preparator and serving layer.

  49. class SimpleEngineParams extends EngineParams

    This shorthand class serves the SimpleEngine class.

  50. abstract class StdevMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Double]

    Returns the global standard deviation of the score returned by the calculate method

  51. abstract class SumMetric[EI, Q, P, A, R] extends Metric[EI, Q, P, A, R] with QPAMetric[Q, P, A, R]

    Returns the sum of the score returned by the calculate method.

  52. class ZeroMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double]

    Returns zero.

  53. trait IEngineFactory extends EngineFactory

    DEPRECATED.

  54. trait IFSPersistentModel[AP <: Params] extends LocalFileSystemPersistentModel[AP]

    DEPRECATED.

  55. trait IFSPersistentModelLoader[AP <: Params, M] extends LocalFileSystemPersistentModelLoader[AP, M]

    DEPRECATED.

  56. trait IPersistentModel[AP <: Params] extends PersistentModel[AP]

    DEPRECATED.

  57. trait IPersistentModelLoader[AP <: Params, M] extends PersistentModelLoader[AP, M]

    DEPRECATED.

  58. class LIdentityPreparator[TD] extends IdentityPreparator[TD]

    DEPRECATED.

  59. class PIdentityPreparator[TD] extends IdentityPreparator[TD]

    DEPRECATED.

  60. trait WithPrId extends AnyRef

    Mix in this trait for queries that contain prId (PredictedResultId).

  61. trait WithQuerySerializer extends CustomQuerySerializer

    DEPRECATED.

Value Members

  1. object Engine extends Serializable

    This object contains concrete implementation for some methods of the Engine class.

  2. object EngineParams extends Serializable

    Companion object for creating EngineParams instances.

  3. object FastEvalEngineWorkflow extends Serializable

    :: Experimental :: Workflow based on FastEvalEngine

  4. object IdentityPreparator extends Serializable

    Companion object of IdentityPreparator that conveniently returns an instance of the class of IdentityPreparator for use with EngineFactory.

  5. object LAverageServing extends Serializable

    A concrete implementation of LServing returning the average of all algorithms' predictions, where their classes are expected to be all Double.

  6. object LFirstServing extends Serializable

    A concrete implementation of LServing returning the first algorithm's prediction result directly without any modification.

  7. object MetricEvaluator extends Serializable

    Companion object of MetricEvaluator

  8. object Utils

    Controller utilities.

  9. object ZeroMetric extends Serializable

    Companion object of ZeroMetric

Deprecated Value Members

  1. object LIdentityPreparator extends Serializable

    DEPRECATED.

  2. object PIdentityPreparator extends Serializable

    DEPRECATED.

Inherited from AnyRef

Inherited from Any

Algorithm

Data Source

Engine

Evaluation

Helper

Preparator

Serving