Pytorch on Veritable Tech Blog

Using ArrayRecord with PyTorch

Tue, 28 Apr 2026 00:00:00 +0000

Cover image generated by ChatGPT

Introduction

ArrayRecord is a new file format developed by Google to “achieve a new frontier of I/O efficiency” [1]. It has been positioned [2] as the successor to TFRecord [3] for storing and feeding data in large-scale machine learning pipelines. It is designed to accommodate three primary access patterns: sequential, batch, and random access. It solves a significant issue in the TFRecord format: the lack of completely random data access, while still providing high I/O performance.

[Notes] Uncovering the Hidden Preprocessing Logic of ColPali

Thu, 22 Jan 2026 00:00:00 +0000

Cover image generated by Nano Banana Pro

Introduction

I recently came across a course called “Multi-Vector Image Retrieval” by DeepLearning.ai. The course mainly introduces ColPali [1], a vision-language model that generalizes the late-interaction retrieval paradigm pioneered by ColBERT [2], extending it from covering only text tokens to covering both text and visual tokens. It also contains a few tutorials on performance optimization techniques using Qdrant’s Python SDK. It is a great introductory resource, and I recommend it to anyone interested in visual document understanding and retrieval.

[Notes] MaxViT: Multi-Axis Vision Transformer

Sun, 16 Jul 2023 00:00:00 +0000

Multi-axis attention to enable both local and global interactions efficiently

[Notes] PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions

Sun, 15 May 2022 00:00:00 +0000

A simple tweak to make your loss function much more adaptable

[Notes] Understanding Visual Attention Network

Mon, 14 Mar 2022 00:00:00 +0000

Decompose large kernel convolutions to get attention weights efficiently

[Notes] Understanding ConvNeXt

Fri, 28 Jan 2022 00:00:00 +0000

A ConvNet for the 2020s

[Notes] Understanding XCiT - Part 2

Sun, 25 Jul 2021 00:00:00 +0000

Local Patch Interaction(LPI) and Class Attention Layer

[Notes] Understanding XCiT - Part 1

Sat, 24 Jul 2021 00:00:00 +0000

Cross-Covariance Attention(XCA) Block

[Notes] Gradient Checkpointing with BERT

Sun, 04 Apr 2021 00:00:00 +0000

A brief analysis of huggingface's implementation

[Paper] Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

Thu, 18 Mar 2021 00:00:00 +0000

Essential for fine-tuning T5 v1.1 and mT5 models

Mistake I Made that Crippled My Streamlit App

Sun, 14 Mar 2021 00:00:00 +0000

Not properly caching slows down the app and increases memory consumption

A Case Study of fastcore @patch_to

Fri, 19 Feb 2021 00:00:00 +0000

Trying out SnapMix with minimal changes to the codebase

[Paper] Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control

Sun, 14 Feb 2021 00:00:00 +0000

Building competitive self-explaining NLP models

[Kaggle] Google Research Football 2020

Mon, 28 Dec 2020 00:00:00 +0000

Describing my 16th place solution and also reviewing some of the others'

[PyTorch Lightning] Log Training Losses when Accumulating Gradients

Tue, 22 Dec 2020 00:00:00 +0000

The global step is not what you think it is

[Tip] TorchScript Supports Half Precision

Sat, 11 Jul 2020 00:00:00 +0000

Speeding up inference for models trained with mixed precision

Self-Supervised Domain Adaptation

Mon, 06 Jul 2020 00:00:00 +0000

Preliminary experiments show great promise

[Failure Report] Distill Fine-tuned Transformers into Recurrent Neural Networks

Tue, 16 Jun 2020 00:00:00 +0000

Transferring knowledge for downstream tasks requires more efforts

Deploying EfficientNet Model using TorchServe

Mon, 04 May 2020 00:00:00 +0000

A Case Study

Fine-tuning BERT for Similarity Search

Thu, 28 Nov 2019 00:00:00 +0000

BERT as Sentence Encoder is Surprisingly Sample-Efficient

Zero Shot Cross-Lingual Transfer with Multilingual BERT

Tue, 24 Sep 2019 00:00:00 +0000

Fine-tuning BERT for Sentence Embeddings on English NLI Datasets

More Memory-Efficient Swish Activation Function

Thu, 22 Aug 2019 00:00:00 +0000

And How to Profile PyTorch GPU Memory Usage

Smaller Docker Image using Multi-Stage Build

Fri, 21 Jun 2019 00:07:25 +0000

Example: CUDA-enabled PyTorch + Apex Image

Mixed Precision Training on Tesla T4 and P100

Thu, 13 Jun 2019 00:07:25 +0000

Training Wide-Resnet with Apex on Google Colab and Kaggle

Use NVIDIA Apex for Easy Mixed Precision Training in PyTorch

Tue, 26 Mar 2019 19:54:30 +0800

Does it benefits those who still use Pascal consumer cards?

Multilingual Similarity Search Using Pretrained Bidirectional LSTM Encoder

Fri, 15 Feb 2019 04:20:35 +0000

Evaluating LASER (Language-Agnostic SEntence Representations)

News Topic Similarity Measure using Pretrained BERT Model

Sun, 10 Feb 2019 04:46:11 +0000

Utilizing Next Sentence Predictions. No Fine-tuning Required.

[Notes] Neural Language Models with PyTorch

Sat, 13 Oct 2018 02:17:58 +0000

With Notebook Examples Runnable on Google Colab

[Learning Note] Single Shot MultiBox Detector with Pytorch — Part 3

Thu, 27 Jul 2017 05:45:21 +0000

Training Objective and Inference

[Learning Note] Single Shot MultiBox Detector with Pytorch — Part 2

Wed, 26 Jul 2017 01:16:40 +0000

Tiling and Matching Strategy

[Learning Note] Single Shot MultiBox Detector with PyTorch — Part 1

Mon, 24 Jul 2017 06:03:30 +0000

An introduction to the Single Shot MultiBox Detector (SSD) with PyTorch, covering its network structure and prediction scheme.