On the Modelling of Impulse Control with Random Effects for Continuous Markov Processes

16 Aug 2020  ·  Helmes K. L., Stockbridge R. H., Zhu C. ·

The use of coordinate processes for the modelling of impulse control for {\em general}\/ Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of an impulse control policy requires that the random times of the interventions be stopping times with respect to different filtrations arising from the different component coordinate processes... When the underlying strong Markov process has {\em continuous}\/ paths, however, a simpler model can be developed which takes the single path space as its probability space and uses the natural filtration with respect to which the intervention times must be stopping times. Moreover, this model construction allows for impulse control with random effects whereby the decision maker selects an impulse but the intervention may result in a different impulse occurring. This paper gives the construction of the probability measure on the path space for an admissible intervention policy subject to a randomized impulse mechanism. It also identifies a class of impulse policies under which the resulting controlled process is Markov and using time-shifts of the policies, a Markov family of time-space dependent measures exists. In addition, a class is defined for which the paths between interventions are independent and a further subclass for which the cycles following the initial cycle are identically distributed. The decision to use an $(s,S)$ ordering policy in inventory management provides an example of an impulse policy for which the process is Markov and has i.i.d.~cycles. A benefit of the constructed model is that one is allowed to use classical renewal arguments to analyze long-term average control problems. read more

PDF Abstract
No code implementations yet. Submit your code now

Categories


Optimization and Control Probability