java.lang.ref


  1. SoftReference: Cleared when GC is response to memory demand. Often used to implement memory-sensitive cache.
  2. WeakReference: Don’t prevent to be finalized. Often used to implement canonicalizing mapping. (Mapping only reachable object instances)
  3. PhantomReference: Are enqueued after determining to be reclaimed. Not automatically cleared by GC. Object referenced via phantom reference won’t be cleared by GC automatically until phantom reference cleared. 

沒有留言:

張貼留言

Lessons Learned While Benchmarking vLLM with GPU

Recently, I benchmarked vLLM on a GPU to better understand how much throughput can realistically be expected in an LLM serving setup. One ...