Go - Cancel Job by close channel


  • Close a channel to broadcast cancellation
  • No way to know a channel is closed or not, so need another channel to indicate a channel is closed
  • This example shows when "done" channel is closed, there is still some work not done, some new messages are sent to channel
    So need consume all messages from channel when a "done channel" is closed



Go - Select Channel


  • Declare
select {
    case <- ch:
        ...
    case x := <- ch2:
        ...
    case ch3 <- y:
        ...
    default:
        ...
}
  • Select wait until some case is ready to proceed
  • Select with no case, wait forever
select {}

Here is example, please notice if more than 1 channel can be process, Go will pick one to process randomly

Lessons Learned While Benchmarking vLLM with GPU

Recently, I benchmarked vLLM on a GPU to better understand how much throughput can realistically be expected in an LLM serving setup. One ...