GSoC 2025 Proposal Submission: Type-Based Alias Analysis in GCC using TySan

[Sumit]

Title: Type-Based Alias Analysis in GCC using TySan

Overview

Both LLVM and GCC share a common sanitizer library called libsanitizer. Recently, libsanitizer has introduced support for Type-Based Sanitization (TySan). The goal of this project is to investigate and prototype the use of type-based aliasing rules information provided by GCC to detect violations of strict aliasing rules.

Objectives

Understand how GCC implements and enforces type-based aliasing rules.
Investigate how aliasing information is represented in GCC’s Intermediate Representation (GIMPLE).
Explore libsanitizer and TySan to identify existing mechanisms for type-based memory access tracking.
Develop a prototype to detect and report violations of strict aliasing rules using type-based aliasing information.
Evaluate the effectiveness of the prototype with test cases demonstrating aliasing rule violations.
Approach

Understanding GCC’s Type-Based Aliasing

Study GCC’s -fstrict-aliasing and -fno-strict-aliasing behavior.
Analyze aliasing information in tree-ssa-alias.c and alias.c.
Investigate how GCC tracks type-based aliasing in its optimization passes.
Exploring libsanitizer and TySan

Study libsanitizer’s existing sanitization techniques.
Identify how TySan detects and reports type-based violations.
Understand how asan, msan, and other sanitizers integrate with GCC.
Prototyping the Detection Mechanism

Modify libsanitizer to extract aliasing information from GCC.
Implement a mechanism to track memory accesses and flag violations when aliasing rules are broken.
Log detected violations for debugging and analysis.
Testing and Evaluation

Develop test cases that violate strict aliasing rules.
Evaluate the effectiveness of the detection mechanism.
Compare performance overhead introduced by the prototype.
Expected Outcome

A working prototype within libsanitizer that detects violations of strict aliasing rules.
Test cases demonstrating the effectiveness of the prototype.
Documentation outlining implementation details and potential improvements.
Benefits to the Community

Helps developers catch aliasing-related undefined behavior early.
Improves debugging of strict aliasing violations in C/C++ programs.
Contributes to the enhancement of GCC’s sanitization tools.
Deliverables

Source code modifications to libsanitizer enabling strict aliasing violation detection.
A set of test cases demonstrating the new functionality.
Documentation and guidelines for integrating the new feature into GCC.
This project aligns with GCC’s goals of improving code correctness and debugging capabilities. I am excited to contribute to GCC’s tooling ecosystem and enhance type-based sanitization capabilities.