EXECUTESAMPLINGOPERATION()`¶

About ExecuteSamplingOperation¶

Submits an asynchronous sampling operation to generate text or compute log-probabilities from the model associated with the specified training session. The session must be in Running or Blocked state. Use GetSamplingOperationResult to poll for the result.

Method Signature¶

METHODS /AWS1/IF_SGI~EXECUTESAMPLINGOPERATION
  IMPORTING
    !IV_TRAININGSESSIONNAME TYPE /AWS1/SGITRAININGSESSIONNAME OPTIONAL
    !IO_OPERATIONCONFIG TYPE REF TO /AWS1/CL_SGISAMPLEOPCONFIG OPTIONAL
  RETURNING
    VALUE(OO_OUTPUT) TYPE REF TO /aws1/cl_sgiexecutesmplopout
  RAISING
    /AWS1/CX_SGICONFLICTEXCEPTION
    /AWS1/CX_SGIINTERNALSERVEREX
    /AWS1/CX_SGIRESOURCENOTFOUNDEX
    /AWS1/CX_SGITHROTTLINGEX
    /AWS1/CX_SGIVLDTNEXCEPTION
    /AWS1/CX_SGICLIENTEXC
    /AWS1/CX_SGISERVEREXC
    /AWS1/CX_RT_TECHNICAL_GENERIC
    /AWS1/CX_RT_SERVICE_GENERIC.

IMPORTING¶

Required arguments:¶

`iv_trainingsessionname` `TYPE /AWS1/SGITRAININGSESSIONNAME` `/AWS1/SGITRAININGSESSIONNAME`¶

The name of the training session to sample from.

`io_operationconfig` `TYPE REF TO /AWS1/CL_SGISAMPLEOPCONFIG` `/AWS1/CL_SGISAMPLEOPCONFIG`¶

The configuration for the sample operation. Specify either a sample configuration to generate text or a computeLogprobs configuration to compute token log-probabilities.

RETURNING¶

`oo_output` `TYPE REF TO /aws1/cl_sgiexecutesmplopout` `/AWS1/CL_SGIEXECUTESMPLOPOUT`¶

Examples¶

Syntax Example¶

This is an example of the syntax for calling the method. It includes every possible argument and initializes every possible value. The data provided is not necessarily semantically accurate (for example the value "string" may be provided for something that is intended to be an instance ID, or in some cases two arguments may be mutually exclusive). The syntax shows the ABAP syntax for creating the various data structures.

DATA(lo_result) = lo_client->executesamplingoperation(
  io_operationconfig = new /aws1/cl_sgisampleopconfig(
    io_computelogprobs = new /aws1/cl_sgicomplogprobsconfig(
      io_prompt = new /aws1/cl_sgimodelinput(
        it_chunks = VALUE /aws1/cl_sgiencodedtextchunk=>tt_encodedtextchunklist(
          (
            new /aws1/cl_sgiencodedtextchunk(
              it_tokens = VALUE /aws1/cl_sgiintegerlist_w=>tt_integerlist(
                ( new /aws1/cl_sgiintegerlist_w( 123 ) )
              )
              iv_type = |string|
            )
          )
        )
      )
      iv_modelpackagearn = |string|
    )
    io_loadweightsforsampling = new /aws1/cl_sgiloadwgtsforsmplcfg( |string| )
    io_sample = new /aws1/cl_sgisampleconfig(
      io_prompt = new /aws1/cl_sgimodelinput(
        it_chunks = VALUE /aws1/cl_sgiencodedtextchunk=>tt_encodedtextchunklist(
          (
            new /aws1/cl_sgiencodedtextchunk(
              it_tokens = VALUE /aws1/cl_sgiintegerlist_w=>tt_integerlist(
                ( new /aws1/cl_sgiintegerlist_w( 123 ) )
              )
              iv_type = |string|
            )
          )
        )
      )
      io_samplingparams = new /aws1/cl_sgisamplingparams(
        it_stop = VALUE /aws1/cl_sgistopsequencelist_w=>tt_stopsequencelist(
          ( new /aws1/cl_sgistopsequencelist_w( |string| ) )
        )
        iv_maxtokens = 123
        iv_seed = 123
        iv_temperature = '0.1'
        iv_topk = 123
        iv_topp = '0.1'
      )
      iv_includepromptlogprobs = ABAP_TRUE
      iv_modelpackagearn = |string|
      iv_numsamples = 123
      iv_topkpromptlogprobs = 123
    )
  )
  iv_trainingsessionname = |string|
).

This is an example of reading all possible response values

lo_result = lo_result.
IF lo_result IS NOT INITIAL.
  lv_operationid = lo_result->get_operationid( ).
ENDIF.

Generate samples from a prompt¶

Generates completions for a tokenized prompt using the session's current weights. Generation behavior is controlled by samplingParams, and the operation is dispatched asynchronously.

DATA(lo_result) = lo_client->executesamplingoperation(
  io_operationconfig = new /aws1/cl_sgisampleopconfig(
    io_sample = new /aws1/cl_sgisampleconfig(
      io_prompt = new /aws1/cl_sgimodelinput(
        it_chunks = VALUE /aws1/cl_sgiencodedtextchunk=>tt_encodedtextchunklist(
          (
            new /aws1/cl_sgiencodedtextchunk(
              it_tokens = VALUE /aws1/cl_sgiintegerlist_w=>tt_integerlist(
                ( new /aws1/cl_sgiintegerlist_w( 791 ) )
                ( new /aws1/cl_sgiintegerlist_w( 6864 ) )
                ( new /aws1/cl_sgiintegerlist_w( 315 ) )
                ( new /aws1/cl_sgiintegerlist_w( 9822 ) )
                ( new /aws1/cl_sgiintegerlist_w( 374 ) )
              )
              iv_type = |encoded_text|
            )
          )
        )
      )
      io_samplingparams = new /aws1/cl_sgisamplingparams(
        iv_maxtokens = 1024
        iv_temperature = '0.7'
        iv_topp = '0.9'
      )
      iv_numsamples = 4
    )
  )
  iv_trainingsessionname = |my-training-session|
).

/AWS1/IF_SGI=>EXECUTESAMPLINGOPERATION()¶