车道线检测

应用场景

• 自动驾驶
• 车道偏移警告
• 自动巡航
• 交通理解

现有方式

1. 传统方式
2. CNN
3. 分割、曲线拟合加上聚类后处理
4. 消息传递和场景信息
5. Soft-attention
6. 目标检测
7. 多项式回归

本文方式

1. Non-local
2. 模型输出为参数
3. Transformer
4. 匈牙利损失

贡献

1. 直接回归车道线和相机姿态
2. 引入Transformer(non-local)
3. 准确率高，资源占用少

模型结构

车道形状模型

匈牙利损失

二分匹配

模型

代码

https://github.com/liuruijin17/LSTR
https://gitee.com/zxysilent/LSTR

# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/kp.py#L150
self.query_embed = nn.Embedding(num_queries, hidden_dim)

# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/kp.py#L181
images = I[0]
# 类别 最小位置 最大位置 X坐标集合 Y坐标集合
masks  = I[1] 
# pmasks Ep
pmasks = F.interpolate(masks[:, 0, :, :][None], size=p.shape[-2:]).to(torch.bool)[0]
# Ell
# p.device
# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/position_encoding.py#L87
pos = self.position_embedding(p, pmasks)

# 方式1
# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/position_encoding.py#L12
class PositionEmbeddingSine(nn.Module):
    def __init__(self, num_pos_feats,temperature,normalize=False, scale=None):
        super().__init__()
        self.num_pos_feats = num_pos_feats
        self.temperature = temperature
        self.normalize = normalize
    def forward(self, x):
# 方式2	
# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/position_encoding.py#L59
class PositionEmbeddingLearned(nn.Module):
    """
    Absolute pos embedding, learned.
    """
    def __init__(self, num_pos_feats=256):
        super().__init__()
        self.row_embed = nn.Embedding(50, num_pos_feats)
        self.col_embed = nn.Embedding(50, num_pos_feats)
    def forward(self, x):
# 方式选择
#
# self._configs["pos_type"] = 'sine'
# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/position_encoding.py#L87
def build_position_encoding(hidden_dim, type):
    N_steps = hidden_dim // 2
    if type in ('v2', 'sine'):
        position_embedding = PositionEmbeddingSine(N_steps, normalize=True)
    elif type in ('v3', 'learned'):
        position_embedding = PositionEmbeddingLearned(N_steps)
    else:
        raise ValueError(f"not supported {type}")
    return position_embedding

def with_pos_embed(self, tensor, pos: Optional[Tensor]):
        return tensor if pos is None else tensor + pos

代码模型

# https://gitee.com/zxysilent/LSTR/blob/culane/models/py_utils/transformer.py#L46
# S,Ep,Sq,Ell
def forward(self, src, mask, query_embed, pos_embed):

实验结果