Enables the first functional 4-bit post-training quantization of the Segment Anything Model (SAM) via two hardware co-designed innovations: Hybrid Log-Uniform Quantization (HLUQ) for heavy-tailed post-GELU activations, and Channel-Aware Grouping (CAG) cutting on-chip register overhead by 99.7%. FPGA deployment achieves 7.89× speedup and 8.64× energy efficiency over float at W4A4 with 36.6% mAP.