Hello,
I am trying to use a model with max_tokens=64000. However, an exception gets thrown, saying that, with a large max_tokens, streaming is strongly recommended. In particular, from where the exception is thrown and from the text of the exception it seems like this should be a warning instead? In my use case, I do not need streaming, and I am fine with not using it, even if the expected time to get a completion is long.
Thank you in advance!
Hello,
I am trying to use a model with
max_tokens=64000. However, an exception gets thrown, saying that, with a largemax_tokens, streaming is strongly recommended. In particular, from where the exception is thrown and from the text of the exception it seems like this should be a warning instead? In my use case, I do not need streaming, and I am fine with not using it, even if the expected time to get a completion is long.Thank you in advance!