#attention-mechanism (23 Repositories)
Ranked open-source repositories tagged with #attention-mechanism, scored by pull request acceptance likelihood and maintainer engagement velocity.
7.2%
14.5h
23 repositories tagged #attention-mechanism
BlinkDL/RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
lucidrains/mimic-video
Implementation of Mimic-Video, Video-Action Models for SOTA Generalizable Robot Control Beyond VLAs
lucidrains/x-transformers
A concise but complete full-attention transformer with a set of promising experimental features from various papers
nndl/nndl
邱锡鹏《神经网络与深度学习》第二版与通识版：电子书、章节目录、学习资源与勘误。
kyegomez/MultiModalMamba
A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.
danielegrattarola/keras-gat
Keras implementation of the graph attention networks (GAT) by Veličković et al. (2017; https://arxiv.org/abs/1710.10903)
EagleW/PaperRobot
Code for PaperRobot: Incremental Draft Generation of Scientific Ideas
sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning
Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
kaushalshetty/Structured-Self-Attention
A Structured Self-attentive Sentence Embedding
kyegomez/PALM-E
Implementation of "PaLM-E: An Embodied Multimodal Language Model"
lucidrains/memorizing-transformers-pytorch
Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch
lucidrains/slot-attention
Implementation of Slot Attention from GoogleAI
husthuaan/AoANet
Code for paper "Attention on Attention for Image Captioning". ICCV 2019
poloclub/dodrio
Exploring attention weights in transformer-based models with linguistic knowledge.
cbaziotis/neat-vision
Neat (Neural Attention) Vision, is a visualization tool for the attention mechanisms of deep-learning models for Natural Language Processing (NLP) tasks. (framework-agnostic)
CASE-Lab-UMD/LLM-Drop
The official implementation of the paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".
Canjie-Luo/MORAN_v2
MORAN: A Multi-Object Rectified Attention Network for Scene Text Recognition
Kyubyong/transformer
A TensorFlow Implementation of the Transformer: Attention Is All You Need
philipturner/metal-flash-attention
FlashAttention (Metal Port)
PetarV-/GAT
Graph Attention Networks (https://arxiv.org/abs/1710.10903)
qinzheng93/GeoTransformer
[CVPR2022] Geometric Transformer for Fast and Robust Point Cloud Registration
atulkum/pointer_summarizer
pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"
justchenhao/STANet
official implementation of the spatial-temporal attention neural network (STANet) for remote sensing image change detection