---
title: "BayesianMaximization"
language: "en"
type: "Symbol"
summary: "BayesianMaximization[f, {conf1, conf2, ...}] gives an object representing the result of Bayesian maximization over the function f over the configurations confi. BayesianMaximization[f, reg] maximizes over the region represented by the region specification reg. BayesianMaximization[f, sampler] maximizes over configurations obtained by applying the function sampler. BayesianMaximization[f, {conf1, conf2, ...} -> nsampler] applies the function nsampler to successively generate configurations starting from the confi."
keywords: 
- maximization
- maximum finding
- bayesian optimization
- black box function
- gaussian process optimization
- surrogate-based optimization
- bayesian search algorithm
- hyperparameter tuning
- expected improvement algorithm
- thompson sampling
canonical_url: "https://reference.wolfram.com/language/ref/BayesianMaximization.html"
source: "Wolfram Language Documentation"
related_guides: 
  - 
    title: "Optimization"
    link: "https://reference.wolfram.com/language/guide/Optimization.en.md"
related_functions: 
  - 
    title: "BayesianMinimization"
    link: "https://reference.wolfram.com/language/ref/BayesianMinimization.en.md"
  - 
    title: "BayesianMaximizationObject"
    link: "https://reference.wolfram.com/language/ref/BayesianMaximizationObject.en.md"
  - 
    title: "NMaximize"
    link: "https://reference.wolfram.com/language/ref/NMaximize.en.md"
  - 
    title: "FindMaximum"
    link: "https://reference.wolfram.com/language/ref/FindMaximum.en.md"
  - 
    title: "Predict"
    link: "https://reference.wolfram.com/language/ref/Predict.en.md"
---
[EXPERIMENTAL]

# BayesianMaximization

BayesianMaximization[f, {conf1, conf2, …}] gives an object representing the result of Bayesian maximization over the function f over the configurations confi.

BayesianMaximization[f, reg] maximizes over the region represented by the region specification reg.

BayesianMaximization[f, sampler] maximizes over configurations obtained by applying the function sampler.

BayesianMaximization[f, {conf1, conf2, …} -> nsampler] applies the function nsampler to successively generate configurations starting from the confi.

## Details and Options

* ``BayesianMaximization````[…]`` returns a ``BayesianMaximizationObject[…]`` whose properties can be obtained using ``BayesianMaximizationObject````[…]["prop"]``.

* Possible properties include:

|                        |                                                             |
| ---------------------- | ----------------------------------------------------------- |
| "EvaluationHistory"    | configurations and values explored during maximization      |
| "MaximumConfiguration" | configuration found that maximizes the result from f        |
| "MaximumValue"         | estimated maximum value obtained from f                     |
| "Method"               | method used for Bayesian maximization                       |
| "NextConfiguration"    | configuration to sample next if maximization were continued |
| "PredictorFunction"    | best prediction model found for the function f              |
| "Properties"           | list of all available properties                            |

* Configurations can be of any form accepted by ``Predict`` (single data element, list of data elements, association of data elements, etc.) and of any type accepted by ``Predict`` (numerical, textual, sounds, images, etc.).

* The function ``f`` must output a real-number value when applied to a configuration ``conf``.

* ``BayesianMaximization[f, …]`` attempts to find a good maximum using the smallest number of evaluations of ``f``.

* In ``BayesianMaximization[f, spec]``, ``spec`` defines the domain of the function ``f``. A domain can be defined by a list of configurations, a geometric region, or a configuration generator function.

* In ``BayesianMaximization[f, sampler]``, ``sampler[]`` must output a configuration suitable for ``f`` to be applied to it.

* In ``BayesianMaximization[f, {conf1, conf2, …} -> nsampler]``, `` nsampler[conf]`` must output a configuration suitable for ``f`` to be applied to it.

* ``BayesianMaximization`` takes the following options:

|                           |           |                                                                   |
| ------------------------- | --------- | ----------------------------------------------------------------- |
| AssumeDeterministic       | False     | whether to assume that f is deterministic                         |
| InitialEvaluationHistory  | None      | initial set of configurations and values                          |
| MaxIterations             | 100       | maximum number of iterations                                      |
| Method                    | Automatic | method used to determine configurations to evaluate               |
| RandomSeeding             | 1234      | what seeding of pseudorandom generators should be done internally |

* Possible settings for ``Method`` include:

|                             |                                                          |
| --------------------------- | -------------------------------------------------------- |
| Automatic                   | automatically choose the method                          |
| "MaxExpectedImprovement"    | maximize expected improvement over current best value    |
| "MaxImprovementProbability" | maximize improvement probability over current best value |

* Possible settings for ``RandomSeeding`` include:

|           |                                                        |
| --------- | ------------------------------------------------------ |
| Automatic | automatically reseed every time the function is called |
| Inherited | use externally seeded random numbers                   |
| seed      | use an explicit integer or strings as a seed           |

---

## Examples (14)

### Basic Examples (3)

Maximize a function over an interval:

```wl
In[1]:= bo = BayesianMaximization[-((# - 2) ^ 2 + 1)&, Interval[{0, 4}]]

Out[1]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 2.0878570008075084, "Value" -> -1.0077188525908904], 
     Association["Configuration" -> 0.3448937078165297, "Value" -> -3.739376838425315],  ... 724492], Association["Configuration" -> 2.0077851684194767, 
      "Value" -> -10.680016342035124]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 12], 
    Association[]]]]
```

Use the resulting ``BayesianMaximizationObject[…]`` to get the estimated maximum configuration:

```wl
In[2]:= bo["MaximumConfiguration"]

Out[2]= 2.00779
```

Get the estimated maximum function value:

```wl
In[3]:= bo["MaximumValue"]

Out[3]= -0.995602
```

---

Maximize a function over a set of configurations:

```wl
In[1]:= bo = BayesianMaximization[(# ^ 2 - 3 * # + 2)&, {5.7, 1.1, 3.4, 6.8, 2.3, 1.2}]

Out[1]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 5.7, "Value" -> 17.39], 
     Association["Configuration" -> 5.7, "Value" -> 17.39], Association["Configuration" -> 2.3, 
      "Value" -> 0.3 ... -> -10.770988120240851], 
     Association["Configuration" -> 5.7, "Value" -> -10.770988120240851]}, 
    TypeSystem`Vector[TypeSystem`Struct[{"Configuration", "Value"}, 
      {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 12], Association[]]]]
```

Get the maximum configuration over the set:

```wl
In[2]:= bo["MaximumConfiguration"]

Out[2]= 5.7
```

---

Maximize a function over a domain defined by a random generator:

```wl
In[1]:= bo = BayesianMaximization[ -(# ^ 2 - 5 * # + 4)&, RandomReal[{0, 5}]&]

Out[1]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 2.6098212510093854, "Value" -> 2.2379392928267343], 
     Association["Configuration" -> 0.43111713477066216, "Value" -> -2.0302763100395547], ... 3908058], Association["Configuration" -> 2.509731460524346, 
      "Value" -> -10.982297213908058]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 12], 
    Association[]]]]
```

Get the estimated maximum value:

```wl
In[2]:= bo["MaximumValue"]

Out[2]= 2.25804
```

### Scope (3)

Maximize a function over a region:

```wl
In[1]:=
f[x_] := -(x - 4) ^ 2 - 1;
reg = Interval[{0., 7.}];

In[2]:= bo = BayesianMaximization[f, reg]

Out[2]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 3.65374975141314, "Value" -> -1.1198892346464624], 
     Association["Configuration" -> 0.6035639886789271, "Value" -> -12.5357775789986], 
   ... 3908058], Association["Configuration" -> 3.513624044734085, 
      "Value" -> -10.871071578797833]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 12], 
    Association[]]]]
```

Get the list of available properties to query:

```wl
In[3]:= bo["Properties"]

Out[3]= {"MaximumConfiguration", "MaximumValue", "EvaluationHistory", "PredictorFunction", "ObjectiveFunction", "Method", "Properties", "NextConfiguration"}
```

Get the history of evaluations:

```wl
In[4]:= bo["EvaluationHistory"]

Out[4]=
Dataset[{Association["Configuration" -> 3.65374975141314, "Value" -> -1.1198892346464624], 
  Association["Configuration" -> 0.6035639886789271, "Value" -> -12.5357775789986], 
  Association["Configuration" -> 5.319272010931116, "Value" -> -2.74047 ... ation["Configuration" -> 4.233838340041474, "Value" -> -1.054680369273352], 
  Association["Configuration" -> 4.080866168610718, "Value" -> -1.006539337225777], 
  Association["Configuration" -> 3.513624044734085, "Value" -> -1.2365615698608314]}]
```

Get information about the method used to determine the configurations to explore:

```wl
In[5]:= bo["Method"]

Out[5]= "MaxExpectedImprovement"
```

Get the current probabilistic model of the function (this is a ``PredictorFunction``):

```wl
In[6]:= p = bo["PredictorFunction"]

Out[6]=
PredictorFunction[Association["ExampleNumber" -> 12, 
  "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", 
      Association["Input" -> Association["f1" -> Association["Type" -> "Numerical"]], 
       "Output" -> A ...  "Date" -> DateObject[{2025, 5, 22, 16, 41, 56.053959`8.501181265413285}, "Instant", 
      "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", 
    "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]]
```

Find the best configuration to explore if the maximization were continued:

```wl
In[7]:= bo["NextConfiguration"]

Out[7]= 4.1156
```

Find a list of properties simultaneously:

```wl
In[8]:= bo[{"ObjectiveFunction", "Method", "MaximumValue"}]

Out[8]= {f, "MaxExpectedImprovement", -0.996902}
```

Visualize how well the function is modeled, particularly near the maximum:

```wl
In[9]:= Plot[{f[{x}], p[{x}]}, {x, 0, 7}, PlotLegends -> {"function", "model"}, Exclusions -> False ]

Out[9]= [image]
```

---

Maximize a function with initial configurations over a domain defined by a random neighborhood configuration generator:

```wl
In[1]:=
f[x_] := -x ^ 2 + 7 * x - 3;
neighborsampler[x_] := RandomReal[{x - 0.2, x + 0.2}];

In[2]:= bo = BayesianMaximization[f, {1., 7., 2.} -> neighborsampler]

Out[2]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 6.9622848051680535, "Value" -> -2.737416072097588], 
     Association["Configuration" -> 1.8292789201031696, "Value" -> 6.458691073188369], 
  ... 3908058], Association["Configuration" -> 3.384509472630643, 
      "Value" -> -10.982297213908058]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 16], 
    Association[]]]]
```

Get the model of the function:

```wl
In[3]:= p = bo["PredictorFunction"]

Out[3]=
PredictorFunction[Association["ExampleNumber" -> 16, 
  "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", 
      Association["Input" -> Association["f1" -> Association["Type" -> "Numerical"]], 
       "Output" -> A ...  "Date" -> DateObject[{2025, 5, 22, 16, 42, 35.887389`8.307516841843706}, "Instant", 
      "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", 
    "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]]
```

Visualize the model's performance near the maximum:

```wl
In[4]:= Plot[{f[{x}], p[{x}]}, {x, 0, 7}, PlotLegends -> {"function", "model"}, Exclusions -> False]

Out[4]= [image]
```

---

Define a function that takes an image and computes the probability returned by ``ImageIdentify`` of identifying the image as the entity Entity["Concept", "Apple::857h7"], with the domain defined by a random generator over a corpus of images:

```wl
In[1]:=
f[image_] := Module[{prob}, prob = ImageIdentify[image, Entity["Concept", "Fruit::855t9"], 5, "Probability"];
	Lookup[prob, Entity["Concept", "Apple::857h7"], 0.]
];
images = WikipediaData["Apple", "ImageList"];
sampler[] := RandomChoice[images];
```

Maximize the function above:

```wl
In[2]:= bo = BayesianMaximization[f, sampler, AssumeDeterministic -> True]

Out[2]= [image]
```

Get the maximum configuration:

```wl
In[3]:= bo["MaximumConfiguration"]

Out[3]= [image]
```

Get the evaluation history:

```wl
In[4]:= bo["EvaluationHistory"]

Out[4]= Dataset[<>]
```

### Options (4)

#### AssumeDeterministic (1)

Maximize a function over a domain defined by a random generator:

```wl
In[1]:=
f[x_] := x Sin[x] - 3 Cos[x] - x / 2;
sampler[] := RandomReal[{-6., 6.}];

In[2]:= bo = BayesianMaximization[f, sampler]

Out[2]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 0.26357100242252507, "Value" -> -2.9595147271663236], 
     Association["Configuration" -> -4.965318876550411, "Value" -> -3.0754049727381734] ... 10283], Association["Configuration" -> -3.2720508671219246, 
      "Value" -> -10.770988120240851]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 13], 
    Association[]]]]
```

The function is assumed to be stochastic; the value from the probabilistic model will differ in general from the function value for the same configuration:

```wl
In[3]:= bo["MaximumValue"]

Out[3]= 5.22272

In[4]:= f[bo["MaximumConfiguration"]]

Out[4]= 5.21414
```

Include information that the function is deterministic, i.e. noise free:

```wl
In[5]:= bo2 = BayesianMaximization[f, sampler, AssumeDeterministic -> True]

Out[5]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 0.26357100242252507, "Value" -> -2.9595147271663236], 
     Association["Configuration" -> -4.965318876550411, "Value" -> -3.0754049727381734] ... 240851], Association["Configuration" -> -2.730282344416005, 
      "Value" -> -10.637456727616328]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 13], 
    Association[]]]]
```

For a deterministic function, the model value and the function value for evaluated configurations agree to a good precision:

```wl
In[6]:= bo2["MaximumValue"]

Out[6]= 5.19275

In[7]:= f[bo2["MaximumConfiguration"]]

Out[7]= 5.21357
```

#### InitialEvaluationHistory (1)

Maximize a function over a disk region with a small number of iterations:

```wl
In[1]:=
f[{x_, y_}] :=   Cos[x] - Exp[(x - 0.5)  * y];
reg = Disk[{0, 0}, 1.];

In[2]:= bo = BayesianMaximization[f, reg, MaxIterations -> 5]

Out[2]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> {-0.8275531460917351, -0.2441740913902284}, 
      "Value" -> -0.7061810180239045], Association["Configuration" -> 
       {0.0875135339021362 ... 62698179645, 0.031979900225265645}, 
      "Value" -> -10.596634733096073]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Vector[TypeSystem`Atom[Real], 2], 
       TypeSystem`Atom[Real]}], 5], Association[]]]]
```

Use the information from this evaluation in the next:

```wl
In[3]:= evalhist = bo["EvaluationHistory"]

Out[3]=
Dataset[{Association["Configuration" -> {-0.8275531460917351, -0.2441740913902284}, 
   "Value" -> -0.7061810180239045], Association["Configuration" -> 
    {0.08751353390213623, -0.041336659055150715}, "Value" -> -0.0210238732443917], 
  Associati ... 42513129729072], Association["Configuration" -> 
    {0.6588747266256556, -0.17082292607260818}, "Value" -> -0.18254386470533424], 
  Association["Configuration" -> {0.9952862698179645, 0.031979900225265645}, 
   "Value" -> -0.47170255481661016]}]

In[4]:= bo2 = BayesianMaximization[f, reg, InitialEvaluationHistory -> evalhist]

Out[4]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> {-0.8275531460917351, -0.2441740913902284}, 
      "Value" -> -0.7061810180239045], Association["Configuration" -> 
       {0.0875135339021362 ...    {-0.27605194658651255, 0.87787887128174}, "Value" -> -10.871071578797833]}, 
    TypeSystem`Vector[TypeSystem`Struct[{"Configuration", "Value"}, 
      {TypeSystem`Vector[TypeSystem`Atom[Real], 2], TypeSystem`Atom[Real]}], 12], Association[]]]]
```

Get the estimated maximum configuration now:

```wl
In[5]:= bo2["MaximumConfiguration"]

Out[5]= {-0.276052, 0.877879}
```

#### MaxIterations (1)

Maximize a function with a domain defined by a random generator:

```wl
In[1]:=
f[x_] := Sin[x] + 4 * Cos[x] + x;
sampler [] := RandomReal[{-5, 5}];

In[2]:= bo = BayesianMaximization[f, sampler]

Out[2]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 0.2196425020187709, "Value" -> 4.341424844259443], 
     Association["Configuration" -> -4.137765730458676, "Value" -> -5.472442943134875], 
  ... 97833], Association["Configuration" -> 0.30342739507070116, 
      "Value" -> -10.680016342035124]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 17], 
    Association[]]]]
```

Get the number of function evaluations:

```wl
In[3]:= Length @ bo["EvaluationHistory"]

Out[3]= 17
```

Specify the maximum number of iterations:

```wl
In[4]:= bo2 = BayesianMaximization[f, sampler, MaxIterations -> 5]

Out[4]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 0.2196425020187709, "Value" -> 4.341424844259443], 
     Association["Configuration" -> -4.137765730458676, "Value" -> -5.472442943134875], 
  ... 0006811], Association["Configuration" -> 0.4292783237337283, 
      "Value" -> -10.871071578797833]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 5], 
    Association[]]]]

In[5]:= Length @ bo2["EvaluationHistory"]

Out[5]= 5
```

#### Method (1)

Define a function over an interval region:

```wl
In[1]:=
f[x_] :=   -x ^ 2 + 3 * x - 2;
int = Interval[{-5., 5.}];
```

Specify the method for exploring configurations:

```wl
In[2]:= bo = BayesianMaximization[f, int, Method -> "MaxExpectedImprovement"]

Out[2]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 0.21964250201877003, "Value" -> -1.3893153226367554], 
     Association["Configuration" -> -4.1377657304586775, "Value" -> -31.534402431534264 ... 3908058], Association["Configuration" -> 1.878314085298622, 
      "Value" -> -10.982297213908058]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 12], 
    Association[]]]]
```

Specify a different method:

```wl
In[3]:= bo2 = BayesianMaximization[f, int, Method -> "MaxImprovementProbability"]

Out[3]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> 0.21964250201877003, "Value" -> -1.3893153226367554], 
     Association["Configuration" -> -4.1377657304586775, "Value" -> -31.534402431534264 ... 630212], Association["Configuration" -> -4.220688895919608, 
      "Value" -> -10.770988120240851]}, TypeSystem`Vector[TypeSystem`Struct[
      {"Configuration", "Value"}, {TypeSystem`Atom[Real], TypeSystem`Atom[Real]}], 12], 
    Association[]]]]
```

### Applications (2)

Define a training set to train predictor functions using the ``Predict`` function and a test set to measure their performance:

```wl
In[1]:=
trainingset = Table[n -> Sin[n], {n, 1, 15}];
testset = {3.9 -> Sin[3.9], 7.3 -> Sin[7.3]};
```

Create a "log-likelihood" function to measure the log-likelihood of test data for different methods in ``Predict`` :

```wl
In[2]:=
func[method_] := PredictorMeasurements[Predict[trainingset, Method -> method], testset, "LogLikelihood"];
methods = {"LinearRegression", "NearestNeighbors", "NeuralNetwork", "GaussianProcess"};
```

Maximize the log-likelihood function over a domain defined by a list of different methods for ``Predict`` :

```wl
In[3]:= bo = BayesianMaximization[func, methods]

Out[3]= BayesianMaximizationObject[«1»]
```

Examine the evaluation history:

```wl
In[4]:= bo["EvaluationHistory"]

Out[4]=
Dataset[{Association["Configuration" -> "GaussianProcess", "Value" -> 5.026287109075449], 
  Association["Configuration" -> "NeuralNetwork", "Value" -> 0.9800545108581262], 
  Association["Configuration" -> "LinearRegression", "Value" -> -2.3167366 ... "Configuration" -> "LinearRegression", "Value" -> -2.3167366185184886], 
  Association["Configuration" -> "NearestNeighbors", "Value" -> -0.15981752491032897], 
  Association["Configuration" -> "LinearRegression", "Value" -> -2.3167366185184886]}]
```

Find the best configuration to explore next:

```wl
In[5]:= bo["NextConfiguration"]

Out[5]= "GaussianProcess"
```

---

Load Fisher's *Iris* dataset and divide it into a training set and a validation set:

```wl
In[1]:=
sampletest  =   RandomSample[ExampleData[{"MachineLearning", "FisherIris"}, "Data"]];
trainingsample = sampletest[[1 ;; 100]];
validationsample = sampletest[[101 ;; ]];
```

Create "blackbox" functions. Here the functions are the log-likelihood functions for two different methods used in the ``Classify`` function. The arguments of the functions are known as hyperparameters.

Train a logistic regression classifier with two hyperparameters, the L1 and L2 regularization coefficients:

```wl
In[2]:=
fLogisticReg[{l1_, l2_}] := Module[
	{class}, 
	class = Classify[trainingsample, Method -> {"LogisticRegression", 
		"L1Regularization" -> Exp[l1], "L2Regularization" -> Exp[l2]}];
	ClassifierMeasurements[class, validationsample, "LogLikelihood"]
];
reg = Rectangle[{-3., -3.}, {3., 3.}];
```

Maximize the log-likelihood function for the logistic regression classifier over a domain defined by a rectangular region in the logarithm of the hyperparameters:

```wl
In[3]:= boLogistic = BayesianMaximization[fLogisticReg, reg]

Out[3]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> {-2.4826594382752054, -0.7325222741706852}, 
      "Value" -> -9.35111158545785], Association["Configuration" -> 
       {-2.93013256218452, 2 ...      {-1.781388745963072, -2.82286130332571}, "Value" -> -2.331766766357626]}, 
    TypeSystem`Vector[TypeSystem`Struct[{"Configuration", "Value"}, 
      {TypeSystem`Vector[TypeSystem`Atom[Real], 2], TypeSystem`Atom[Real]}], 12], Association[]]]]
```

Get the model of the function:

```wl
In[4]:= pLogistic = boLogistic["PredictorFunction"]

Out[4]=
PredictorFunction[Association["ExampleNumber" -> 12, 
  "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", 
      Association["Input" -> Association["f1" -> Association["Type" -> "NumericalVector", 
           "Leng ...   "Date" -> DateObject[{2025, 5, 22, 17, 2, 57.732684`8.513996721988045}, "Instant", "Gregorian", 
      -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", 
    "SystemWordLength" -> 64, "Evaluations" -> {}]]]
```

Visualize the model of the log-likelihood function of the classifier together with the estimated maximum:

```wl
In[5]:= pt = {Append[boLogistic["MaximumConfiguration"], boLogistic["MaximumValue"]]}

Out[5]= {{-2.56911, -2.99273, -6.41303}}

In[6]:= Show[Plot3D[pLogistic[{x, y}], {x, -3, 3}, {y, -3, 3}, PlotLegends -> {"Logistic Regression"}, Exclusions -> False], ListPointPlot3D[pt, PlotStyle -> Directive[Red, PointSize[Large]]]]

Out[6]= [image]
```

Now train a support vector machine (SVM) classifier with two hyperparameters, the soft margin parameter and the gamma scaling parameter:

```wl
In[7]:=
fSVMRBF[{c_, gamma_}] := Module[
	{class}, 
	class  =   Classify[trainingsample, Method -> {"SupportVectorMachine", "KernelType" -> 
		"RadialBasisFunction", "SoftMarginParameter" -> Exp[c], "GammaScalingParameter" -> Exp[gamma]}];
	ClassifierMeasurements[class, validationsample, "LogLikelihood"]
];
reg2 = Rectangle[{-3., -3.}, {3., 3.}];
```

Maximize the log-likelihood function for the SVM classifier over a domain defined by a rectangular region in the logarithm of the hyperparameters:

```wl
In[8]:= boSVM = BayesianMaximization[fSVMRBF, reg2]

Out[8]=
BayesianMaximizationObject[Association["EvaluationHistory" -> 
   Dataset[{Association["Configuration" -> {-2.4826594382752054, -0.7325222741706852}, 
      "Value" -> -13.155032850815166], Association["Configuration" -> 
       {-2.93013256218452, ...      {2.370681784806137, 1.2310114246669275}, "Value" -> -2.499342344864331]}, 
    TypeSystem`Vector[TypeSystem`Struct[{"Configuration", "Value"}, 
      {TypeSystem`Vector[TypeSystem`Atom[Real], 2], TypeSystem`Atom[Real]}], 12], Association[]]]]
```

Get the model of the function:

```wl
In[9]:= pSVM = boSVM["PredictorFunction"]

Out[9]=
PredictorFunction[Association["ExampleNumber" -> 12, 
  "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", 
      Association["Input" -> Association["f1" -> Association["Type" -> "NumericalVector", 
           "Leng ...    "Date" -> DateObject[{2025, 5, 22, 17, 3, 9.641604`7.736724276873677}, "Instant", "Gregorian", 
      -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", 
    "SystemWordLength" -> 64, "Evaluations" -> {}]]]
```

Visualize the model of the log-likelihood function of the classifier together with the estimated maximum:

```wl
In[10]:= pt2 = {Append[boSVM["MaximumConfiguration"], boSVM["MaximumValue"]]}

Out[10]= {{1.11402, -2.69924, -5.66007}}

In[11]:= Show[Plot3D[pSVM[{x, y}], {x, -3, 3}, {y, -3, 3}, PlotLegends -> "Support Vector Machine-RBF", ColorFunction -> "BlueGreenYellow", Exclusions -> False], ListPointPlot3D[pt2, PlotStyle -> Directive[Red, PointSize[Large]]]]

Out[11]= [image]
```

### Possible Issues (2)

When the domain of the objective function is defined by an initial configuration set and a neighborhood configuration generator, the results depend on the quality of the generator provided.

Minimize a function where the domain is defined as above:

```wl
In[1]:=
f[x_] := Sin[x] - x * Cos[x];
ns[x_] := RandomReal[{x - 0.1, x + 0.1}];

In[2]:= bo = BayesianMaximization[f, {0, 3.} -> ns, Method -> Automatic, AssumeDeterministic -> True]

Out[2]= BayesianMaximizationObject[«1»]
```

Get the model of the function:

```wl
In[3]:= p = bo["PredictorFunction"]

Out[3]=
PredictorFunction[Association["ExampleNumber" -> 30, 
  "Input" -> Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", 
      Association["Input" -> Association["f1" -> Association["Type" -> "Numerical"]], 
       "Output" -> A ...     "Date" -> DateObject[{2025, 5, 22, 17, 4, 9.03776`7.708635791053802}, "Instant", "Gregorian", 
      -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", 
    "SystemWordLength" -> 64, "Evaluations" -> {}]]]
```

Since the starting initial configurations are "far" from the global maximum and the generator takes relatively small steps, the algorithm could converge to a local maximum:

```wl
In[4]:= Plot[{f[x], p[x]}, {x, -8, 8}, Exclusions -> False]

Out[4]= [image]
```

---

If the neighborhood configuration generator takes steps that are "too small" or "too large", this could lead to issues:

```wl
In[1]:=
SeedRandom[2];
(bo = BayesianMaximization[# ^ 2 &, {1., 2., 3.} -> (RandomReal[{# - 0.01, # + 0.01}]&), AssumeDeterministic -> True])//AbsoluteTiming

Out[1]= {49.7276, BayesianMaximizationObject[«1»]}
```

This takes a rather long time to evaluate. Since the generator takes very small steps each time, the algorithm keeps running until reaches its default value of 100 iterations:

```wl
In[2]:= bo["EvaluationHistory"]

Out[2]=
Dataset[{Association["Configuration" -> 1.9981142402584027, "Value" -> 3.9924605171234138], 
  Association["Configuration" -> 2.9914639460051586, "Value" -> 8.948856540248755], 
  Association["Configuration" -> 3.017683992156856, "Value" -> 9.10641 ... ociation["Configuration" -> 4.957053712314155, "Value" -> 24.57238150676755], 
  Association["Configuration" -> 4.975429243151218, "Value" -> 24.754896153604307], 
  Association["Configuration" -> 4.99712595522803, "Value" -> 24.971267812413647]}]
```

## See Also

* [`BayesianMinimization`](https://reference.wolfram.com/language/ref/BayesianMinimization.en.md)
* [`BayesianMaximizationObject`](https://reference.wolfram.com/language/ref/BayesianMaximizationObject.en.md)
* [`NMaximize`](https://reference.wolfram.com/language/ref/NMaximize.en.md)
* [`FindMaximum`](https://reference.wolfram.com/language/ref/FindMaximum.en.md)
* [`Predict`](https://reference.wolfram.com/language/ref/Predict.en.md)

## Related Guides

* [`Optimization`](https://reference.wolfram.com/language/guide/Optimization.en.md)

## History

* [Introduced in 2016 (11.0)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn110.en.md) \| [Updated in 2017 (11.2)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn112.en.md)