K100_AI单卡全离线部署智能会议记录系统
·
一、引言
会议记录是企业日常沟通与决策的重要载体，然而传统人工记录耗时费力，且容易遗漏关键信息。随着人工智能技术的发展，语音识别（ASR）与声纹识别技术逐步成熟，使得自动生成带说话人标注的会议纪要成为可能。但当前多数方案依赖云端API，存在网络延迟、数据隐私与长期使用成本三大痛点。对于政务、金融、医疗等对数据安全要求高的场景，全离线部署是刚性需求。
K100_AI加速卡的推出，使得在单张加速卡上同时承载大语言模型与大参数量语音模型成为现实。本文设计并实现了一套基于Qwen3-ASR-1.7B语音识别模型与ERes2Net V2声纹识别模型的智能会议记录系统，全部推理工作均在一张K100_AI单卡上完成，无需任何互联网连接。系统采用Flask+Socket.IO框架实现前端网页与后端服务的实时双向通信，前端通过浏览器麦克风采集音频，后端进行低延迟的语音活动检测（VAD）、流式语音分割、ASR转写与说话人识别，最终在前端界面实时呈现，并支持一键保存全文记录。
本文在保证转写准确率的同时，充分考虑了单卡资源的合理调度：Qwen3-ASR模型使用vLLM部署为高性能推理API服务，声纹模型通过ModelScope本地加载并常驻显存。整个系统安装部署简单，所有模型与服务均封装在单节点内，真正做到了开箱即用的全离线会议记录体验。
二、方案设计
系统整体采用“浏览器采集—服务端处理—浏览器展示”的B/S架构，核心服务均部署于配备单张K100_AI加速卡的服务器上。功能模块如下：
1、前端采集与交互：基于HTML5-WebAudio-API实时捕获麦克风音频，以16kHz单声道格式分帧发送至后端；同时提供会议控制、噪音校准、声纹注册/测试、阈值调节和记录保存等交互功能。
2、后端核心服务：使用Python Flask与Flask-Socket.IO构建，负责接收音频流、执行两级VAD（基于能量与静音时长）、语音段切割、调用ASR服务转写文本，并通过声纹模型标注说话人。识别结果通过WebSocket实时推送至前端。
3、ASR推理服务：Qwen3-ASR-1.7B模型以兼容OpenAIAPI的HTTP服务形式部署在K100_AI上（使用vLLM或FastAPI封装），后端通过REST调用完成语音片段转录。配置文件中的API地址可指向localhost的本机服务，实现全离线。
4、声纹识别模块：加载iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common模型，通过提取说话人嵌入向量并与注册声纹库比对完成身份确认。对输入音频进行去静音、过滤过短片段和动态阈值匹配，有效提升识别鲁棒性。
5、并发与缓冲设计：采用线程池异步处理语音段，避免阻塞主线程；设置语音活动缓冲区和分段逻辑，确保长句与停顿的正确切分。
下图展示了整体数据流：
单卡资源调度策略上，ASR服务占据大部分计算与显存，声纹模型轻量常驻，两者分时复用GPU，避免了资源冲突。
三、实施方法及代码
3.1硬件环境
本方案的硬件平台为一台H3C服务器，配置如下：
|
组件 |
规格 |
|
CPU |
2×海光74902.7GHz64C |
|
内存 |
16×32GDDR5 |
|
GPU |
8×海光DCU64GBK100_AI |
只需占用一张K100_AI显卡
3.2软件栈
本方案的软件栈基于Docker容器化技术构建，使用经过海光DCU适配的vLLM推理镜像：
镜像（Qwen3-ASR-1.7B）：
harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220
该镜像基于vLLM0.15.1推理框架、DTK26.04，Python3.10环境，单卡部署Qwen3-ASR-1.7B，200ms即可完成语音识别。
软件项目：
1、Qwen3-ASR-1.7B环境部署参照https://developer.sourcefind.cn/codes/modelzoo/qwen3-asr_pytorch里面操作步骤，项目下载链接为：https://developer.sourcefind.cn/codes/modelzoo/qwen3-asr_pytorch/-/archive/main/qwen3-asr_pytorch-main.zip。
2、智能会议记录系统自编程序代码如下：
#!/usr/bin/env python3
"""
智能会议记录系统 — Flask + SocketIO 实时版（Qwen3-ASR API）
修复：正确处理英文语言标识，避免吞字
优化：声纹识别增加去静音、短片段过滤、动态阈值
新增：声纹测试功能、会议记录保存到本地文件
"""
import eventlet
import sys
if 'gunicorn' not in sys.modules:
eventlet.monkey_patch()
import os
import io
import wave
import json
import time
import base64
import struct
import threading
import tempfile
import re
from collections import defaultdict
from concurrent.futures import ThreadPoolExecutor
import numpy as np
import torch
import torchaudio
import soundfile as sf
from scipy import signal
import requests
from flask import Flask, render_template_string, request
from flask_socketio import SocketIO, emit
from modelscope.models import Model
# ================== 配置 ==================
DEFAULT_CONFIG = {
"asr_api_url": "http://192.168.222.65:8084/v1/audio/transcriptions",
"asr_model": "Qwen3-ASR-1.7B",
"asr_api_key": "PassWord@123456",
"asr_sample_rate": 16000,
"asr_frame_duration": 30,
"asr_vad_mode": 1,
"asr_silence_thresh": 15,
"vad_energy_threshold": 0.02, # 修改为 0.02
"vad_silence_duration": 0.9, # 修改为 0.9
"vad_max_speech_sec": 15,
"noise_words": [
"ok.", "Yeah.", "hmm", "uh", "um", "ah", "oh", "mhm",
"嗯。", "啊", "哦"
]
}
SPEAKER_MODEL_ID = "iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common"
SPEAKER_DB_PATH = "speaker_db.json"
SAMPLE_RATE = 16000
DEFAULT_ENERGY_THRESHOLD = DEFAULT_CONFIG["vad_energy_threshold"]
DEFAULT_SILENCE_DURATION = DEFAULT_CONFIG["vad_silence_duration"]
MAX_SPEECH_SEC = 5
BUFFER_TIMEOUT = 0.1
# 声纹识别动态阈值参数
SPEAKER_ABS_THRESH = 0.55
SPEAKER_MARGIN = 0.12
MIN_SPEECH_DURATION = 1.0
NOISE_WORDS_SET = set(word.lower().strip() for word in DEFAULT_CONFIG["noise_words"])
executor = ThreadPoolExecutor(max_workers=4)
# ================== 声纹管理 ==================
class SpeakerManager:
def __init__(self, db_path=SPEAKER_DB_PATH):
self.db_path = db_path
self.db = self.load_db()
self.lock = threading.Lock()
self.device = "cuda:6" if torch.cuda.is_available() else "cpu"
self.speaker_model = Model.from_pretrained(
SPEAKER_MODEL_ID,
revision=None,
device=self.device,
)
self.speaker_model.eval()
print("✅ 声纹模型加载成功", flush=True)
def load_db(self):
if os.path.exists(self.db_path):
with open(self.db_path, 'r', encoding='utf-8') as f:
return json.load(f)
return {}
def save_db(self):
with open(self.db_path, 'w', encoding='utf-8') as f:
json.dump(self.db, f, ensure_ascii=False, indent=2)
def extract_embedding_from_array(self, audio_array, sr=16000):
if audio_array is None or len(audio_array) == 0:
return None
try:
data = audio_array.astype(np.float32)
if data.ndim > 1:
data = np.mean(data, axis=1)
waveform = torch.from_numpy(data).unsqueeze(0)
if sr != 16000:
resampler = torchaudio.transforms.Resample(orig_freq=sr, new_freq=16000)
waveform = resampler(waveform)
waveform = waveform.to(self.device)
with torch.no_grad():
embedding = self.speaker_model(waveform)
if isinstance(embedding, torch.Tensor):
embedding = embedding.squeeze().cpu().tolist()
return embedding
except Exception as e:
import traceback
traceback.print_exc()
return None
def register_speaker(self, name, audio_path):
if not os.path.isfile(audio_path):
return False, "音频文件无效"
data, sr = sf.read(audio_path, dtype='float32')
if data.ndim > 1:
data = np.mean(data, axis=1)
embedding = self.extract_embedding_from_array(data, sr)
if embedding is None:
return False, "无法提取声纹特征"
with self.lock:
self.db[name] = embedding
self.save_db()
return True, f"说话人 '{name}' 注册成功！"
def delete_speaker(self, name):
if not name or not name.strip():
return False, "说话人姓名不能为空"
with self.lock:
if name not in self.db:
return False, f"说话人 '{name}' 不存在"
del self.db[name]
self.save_db()
return True, f"说话人 '{name}' 已删除"
def identify_speaker(self, audio_array, sr=16000):
if not self.db:
return None, 0.0, "声纹数据库为空"
embedding = self.extract_embedding_from_array(audio_array, sr)
if embedding is None:
return None, 0.0, "无法提取声纹特征"
emb_tensor = torch.tensor(embedding)
with self.lock:
db_copy = self.db.copy()
scores = []
for name, db_emb in db_copy.items():
db_tensor = torch.tensor(db_emb)
similarity = torch.nn.functional.cosine_similarity(
emb_tensor.unsqueeze(0), db_tensor.unsqueeze(0)
).item()
scores.append((similarity, name))
scores.sort(reverse=True, key=lambda x: x[0])
if not scores:
return None, 0.0, "数据库为空"
best_score, best_name = scores[0]
second_score = scores[1][0] if len(scores) > 1 else -1.0
if best_score >= SPEAKER_ABS_THRESH and (best_score - second_score) >= SPEAKER_MARGIN:
return best_name, best_score, f"识别成功: {best_name} ({best_score:.3f})"
else:
return None, best_score, f"未匹配 (best={best_score:.3f}, 2nd={second_score:.3f})"
def get_speaker_list(self):
with self.lock:
return list(self.db.keys())
speaker_mgr = SpeakerManager()
# ================== 工具函数 ==================
def trim_silence(audio_np, sr=SAMPLE_RATE, top_db=30):
if len(audio_np) < sr * 0.5:
return audio_np
frame_len = int(sr * 0.025)
hop = int(sr * 0.010)
energy = np.array([
np.sum(np.abs(audio_np[i:i+frame_len]**2))
for i in range(0, len(audio_np)-frame_len, hop)
])
if len(energy) == 0 or energy.max() < 1e-10:
return audio_np
threshold = 0.01 * energy.max()
mask = energy > threshold
if not mask.any():
return audio_np
start_idx = np.argmax(mask) * hop
end_idx = (len(mask) - np.argmax(mask[::-1]) - 1) * hop + frame_len
return audio_np[start_idx:max(end_idx, start_idx + int(sr*0.2))]
def pcm_to_wav_bytes(pcm_data, sample_rate, num_channels=1, sampwidth=2):
wav_io = io.BytesIO()
with wave.open(wav_io, 'wb') as wf:
wf.setnchannels(num_channels)
wf.setsampwidth(sampwidth)
wf.setframerate(sample_rate)
wf.writeframes(pcm_data)
return wav_io.getvalue()
def is_noise_text(text):
cleaned = text.strip().lower().rstrip('.!?,;:。！？，；：')
return cleaned in NOISE_WORDS_SET
def is_punctuation_only(text):
cleaned = re.sub(r'[^\u4e00-\u9fff\u3400-\u4dbfa-zA-Z0-9]', '', text)
return len(cleaned) == 0
# ================== ASR 调用 ==================
class ASRGenerator:
def __init__(self, config):
self.api_url = config["asr_api_url"]
self.api_key = config["asr_api_key"]
self.model = config["asr_model"]
self.sample_rate = config["asr_sample_rate"]
self.frame_duration = config["asr_frame_duration"]
self.vad_mode = config["asr_vad_mode"]
self.silence_thresh = config["asr_silence_thresh"]
self.session = requests.Session()
if self.api_key:
self.session.headers.update({'Authorization': f'Bearer {self.api_key}'})
def transcribe_pcm(self, pcm_bytes):
if not pcm_bytes:
return ""
wav_bytes = pcm_to_wav_bytes(pcm_bytes, self.sample_rate)
try:
files = {'file': ('audio.wav', io.BytesIO(wav_bytes), 'audio/wav')}
data = {
'model': self.model,
'language': 'zh',
'response_format': 'json',
'sample_rate': self.sample_rate,
'frame_duration': self.frame_duration,
'vad_mode': self.vad_mode,
'silence_thresh': self.silence_thresh
}
response = self.session.post(self.api_url, files=files, data=data, timeout=30)
response.raise_for_status()
result = response.json()
if 'text' in result:
raw_text = result['text'].strip()
elif 'choices' in result and len(result['choices']) > 0:
raw_text = result['choices'][0].get('text', '').strip()
else:
raw_text = ""
clean_text = re.sub(r'<[^>]*>', ' ', raw_text)
clean_text = re.sub(r'\blanguage\s+[A-Za-z]+\s*', '', clean_text, flags=re.IGNORECASE)
clean_text = re.sub(r'\s+', ' ', clean_text).strip()
return clean_text
except Exception as e:
print(f"[ASR] 请求错误: {e}", flush=True)
return ""
asr_gen = ASRGenerator(DEFAULT_CONFIG)
print("✅ ASR API 就绪", flush=True)
# ================== Flask 应用 ==================
app = Flask(__name__)
app.secret_key = os.urandom(24).hex()
socketio = SocketIO(app, async_mode='eventlet', cors_allowed_origins="*",
ping_timeout=60, ping_interval=25,
max_http_buffer_size=10 * 1024 * 1024)
sessions = defaultdict(lambda: {
'vad': {
'voiced_frames': b'',
'silence_duration': 0.0,
'speaking': False,
'speech_duration': 0.0,
'energy_threshold': DEFAULT_ENERGY_THRESHOLD,
'silence_thresh': DEFAULT_SILENCE_DURATION
},
'recording': False,
'stop_event': threading.Event(),
'lock': threading.Lock(),
'transcription': [],
'noise_threshold': None,
'noise_filter_enabled': False,
'debug_counter': 0
})
def process_segment(sid, pcm_bytes):
text = asr_gen.transcribe_pcm(pcm_bytes)
if not text or is_noise_text(text) or is_punctuation_only(text):
return
audio_np = np.frombuffer(pcm_bytes, dtype=np.int16).astype(np.float32) / 32768.0
duration = len(audio_np) / SAMPLE_RATE
speaker = "未知说话人"
if duration >= MIN_SPEECH_DURATION:
trimmed_audio = trim_silence(audio_np, sr=SAMPLE_RATE)
name, score, msg = speaker_mgr.identify_speaker(trimmed_audio, sr=SAMPLE_RATE)
if name:
speaker = name
else:
speaker = f"未知说话人({score:.3f})"
else:
print(f"[SEGMENT] 语音片段过短 ({duration:.2f}s)，跳过声纹识别", flush=True)
record = {
"time": time.strftime("%H:%M:%S"),
"speaker": speaker,
"text": text.strip()
}
sessions[sid]['transcription'].append(record)
socketio.emit('transcription', record, room=sid)
# ---------- SocketIO 事件 ----------
@socketio.on('connect')
def on_connect():
emit('connected', {'sid': request.sid})
@socketio.on('disconnect')
def on_disconnect():
sessions.pop(request.sid, None)
@socketio.on('clear_audio_state')
def on_clear_audio_state():
sid = request.sid
sessions[sid]['vad'] = {
'voiced_frames': b'',
'silence_duration': 0.0,
'speaking': False,
'speech_duration': 0.0,
'energy_threshold': DEFAULT_ENERGY_THRESHOLD,
'silence_thresh': DEFAULT_SILENCE_DURATION
}
@socketio.on('start_recording')
def on_start_recording():
sid = request.sid
sessions[sid]['recording'] = True
on_clear_audio_state()
sessions[sid]['transcription'] = []
@socketio.on('stop_recording')
def on_stop_recording():
sid = request.sid
sessions[sid]['recording'] = False
vad = sessions[sid]['vad']
if vad['voiced_frames']:
executor.submit(process_segment, sid, vad['voiced_frames'])
vad['voiced_frames'] = b''
@socketio.on('set_noise_threshold')
def on_set_noise_threshold(data):
sid = request.sid
rms = data.get('rms', 0.0)
if rms > 0:
sessions[sid]['noise_threshold'] = rms * 2.0
sessions[sid]['noise_filter_enabled'] = True
new_energy = max(rms * 8.0, 0.02)
sessions[sid]['vad']['energy_threshold'] = new_energy
socketio.emit('update_energy_threshold', {'value': round(new_energy, 4)}, room=sid)
emit('log', {'msg': f'✅ 噪音阈值已校准: {rms*2:.6f}，能量阈值自动调整为: {new_energy:.4f}'}, room=sid)
@socketio.on('set_silence_duration')
def on_set_silence_duration(data):
sid = request.sid
duration = float(data.get('duration', DEFAULT_SILENCE_DURATION))
if duration < 0.1:
duration = 0.1
elif duration > 5.0:
duration = 5.0
sessions[sid]['vad']['silence_thresh'] = duration
emit('log', {'msg': f'✅ 静音检测时间已设置为 {duration:.1f} 秒'}, room=sid)
@socketio.on('set_energy_threshold')
def on_set_energy_threshold(data):
sid = request.sid
threshold = float(data.get('threshold', DEFAULT_ENERGY_THRESHOLD))
if threshold < 0.001:
threshold = 0.001
elif threshold > 0.5:
threshold = 0.5
sessions[sid]['vad']['energy_threshold'] = threshold
emit('log', {'msg': f'✅ 能量阈值已设置为 {threshold:.4f}'}, room=sid)
@socketio.on('audio_chunk')
def on_audio_chunk(data):
sid = request.sid
session = sessions[sid]
if not session['recording']:
return
if isinstance(data, bytearray):
raw = bytes(data)
else:
raw = data
if len(raw) < 4:
return
original_rate = struct.unpack_from('<I', raw, 0)[0]
float_data = raw[4:]
if len(float_data) == 0:
return
samples = np.frombuffer(float_data, dtype=np.float32).copy()
if original_rate != SAMPLE_RATE:
samples = signal.resample_poly(samples, SAMPLE_RATE, original_rate)
samples = np.clip(samples, -1.0, 1.0)
vad = session['vad']
threshold = vad['energy_threshold']
frame_duration = len(samples) / SAMPLE_RATE
rms = np.sqrt(np.mean(samples ** 2))
noise_thresh = session.get('noise_threshold')
if noise_thresh and session.get('noise_filter_enabled'):
if rms < noise_thresh and not vad['speaking']:
session['debug_counter'] += 1
return
int_samples = (samples * 32767).astype(np.int16)
pcm_frame = int_samples.tobytes()
socketio.emit('rms_update', {'rms': round(float(rms), 6)}, room=sid)
silence_thresh = vad['silence_thresh']
if rms >= threshold:
if not vad['speaking']:
vad['speaking'] = True
vad['voiced_frames'] += pcm_frame
vad['silence_duration'] = 0.0
vad['speech_duration'] += frame_duration
else:
if vad['speaking']:
vad['voiced_frames'] += pcm_frame
vad['silence_duration'] += frame_duration
if (vad['silence_duration'] >= silence_thresh or
vad['speech_duration'] >= MAX_SPEECH_SEC):
segment = vad['voiced_frames']
vad['voiced_frames'] = b''
vad['silence_duration'] = 0.0
vad['speaking'] = False
vad['speech_duration'] = 0.0
if segment:
executor.submit(process_segment, sid, segment)
# 声纹管理事件
@socketio.on('register_speaker')
def on_register_speaker(data):
name = data.get('name', '').strip()
audio_b64 = data.get('audio', '')
if not name or not audio_b64:
emit('register_result', {'success': False, 'msg': '姓名或音频为空'})
return
try:
audio_bytes = base64.b64decode(audio_b64)
except Exception as e:
emit('register_result', {'success': False, 'msg': f'音频解码失败: {str(e)}'})
return
tmp = tempfile.NamedTemporaryFile(suffix='.wav', delete=False)
tmp.write(audio_bytes)
tmp.close()
success, msg = speaker_mgr.register_speaker(name, tmp.name)
os.unlink(tmp.name)
emit('register_result', {'success': success, 'msg': msg})
@socketio.on('delete_speaker')
def on_delete_speaker(data):
name = data.get('name', '').strip()
success, msg = speaker_mgr.delete_speaker(name)
emit('delete_result', {'success': success, 'msg': msg})
@socketio.on('get_speaker_list')
def on_get_speaker_list():
emit('speaker_list', {'speakers': speaker_mgr.get_speaker_list()})
@socketio.on('test_speaker')
def on_test_speaker(data):
audio_b64 = data.get('audio', '')
if not audio_b64:
emit('test_result', {'success': False, 'msg': '音频数据为空'})
return
try:
audio_bytes = base64.b64decode(audio_b64)
except Exception as e:
emit('test_result', {'success': False, 'msg': f'音频解码失败: {str(e)}'})
return
tmp = tempfile.NamedTemporaryFile(suffix='.wav', delete=False)
tmp.write(audio_bytes)
tmp.close()
try:
data_np, sr = sf.read(tmp.name, dtype='float32')
if data_np.ndim > 1:
data_np = np.mean(data_np, axis=1)
name, score, msg = speaker_mgr.identify_speaker(data_np, sr)
emit('test_result', {
'success': True,
'name': name if name else '未知',
'score': round(score, 4),
'msg': msg
})
except Exception as e:
emit('test_result', {'success': False, 'msg': f'识别处理失败: {str(e)}'})
finally:
os.unlink(tmp.name)
# ================== HTML 模板（含保存功能） ==================
HTML_TEMPLATE = '''
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<title>智能会议记录系统</title>
<style>
:root {
--bg: #f4f7f9;
--panel: rgba(255,255,255,0.95);
--accent: #1e90ff;
--text: #2c3e50;
}
* { margin:0; padding:0; box-sizing:border-box; }
body {
background: var(--bg);
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Microsoft YaHei', sans-serif;
color: var(--text);
height: 100vh;
display: flex;
flex-direction: column;
overflow: hidden;
}
.top-bar {
padding: 12px 20px;
background: rgba(255,255,255,0.9);
border-bottom: 1px solid #ccc;
display: flex;
justify-content: space-between;
align-items: center;
flex-shrink: 0;
}
.top-bar .logo { font-size: 1.2em; font-weight: bold; color: var(--accent); }
.main {
display: flex;
flex: 1;
min-height: 0;
}
.sidebar {
width: 280px;
background: rgba(255,255,255,0.8);
border-right: 1px solid #ddd;
padding: 15px;
overflow-y: auto;
display: flex;
flex-direction: column;
gap: 12px;
flex-shrink: 0;
}
.content {
flex: 1;
display: flex;
flex-direction: column;
min-height: 0;
}
.transcript-area {
flex: 1;
overflow-y: auto;
padding: 15px;
background: white;
margin: 10px;
border-radius: 10px;
box-shadow: 0 2px 8px rgba(0,0,0,0.05);
}
.controls {
padding: 10px 15px;
display: flex;
gap: 10px;
align-items: center;
}
button {
padding: 10px 18px;
border: none;
border-radius: 8px;
background: var(--accent);
color: white;
font-size: 1em;
cursor: pointer;
transition: 0.2s;
}
button:disabled { opacity: 0.5; cursor: default; }
button.secondary { background: #95a5a6; }
button.danger { background: #e74c3c; }
.msg-row {
margin-bottom: 8px;
padding: 6px 10px;
background: #f8f9fa;
border-radius: 6px;
}
.speaker { font-weight: bold; color: #2e86de; }
.text { margin-left: 8px; }
input, select {
padding: 8px;
border: 1px solid #ddd;
border-radius: 6px;
width: 100%;
}
.section-title { font-size: 0.9em; font-weight: bold; margin-top: 8px; color: #555; }
.slider-container {
display: flex;
flex-direction: column;
gap: 4px;
margin-bottom: 8px;
}
.slider-container label {
font-size: 0.85em;
color: #555;
}
.slider-container input[type=range] {
width: 100%;
}
.slider-value {
font-weight: bold;
color: var(--accent);
}
.rms-bar {
width: 100%;
height: 8px;
background: #eee;
border-radius: 4px;
margin-top: 2px;
}
.rms-fill {
height: 100%;
background: #2ecc71;
border-radius: 4px;
transition: width 0.1s;
}
</style>
</head>
<body>
<div class="top-bar">
<span class="logo">🎤 智能会议记录</span>
<span id="status">⚪ 未连接</span>
</div>
<div class="main">
<div class="sidebar">
<div class="section-title">⚙️ 语音检测</div>
<div class="slider-container">
<label>能量阈值 <span id="energyValue">0.0200</span></label>
<input type="range" id="energySlider" min="0.001" max="0.5" step="0.001" value="0.02" oninput="updateEnergy(this.value)">
<div class="rms-bar"><div id="rmsFill" class="rms-fill" style="width:0%"></div></div>
<span style="font-size:0.75em;">实时能量: <span id="rmsText">0.000000</span></span>
</div>
<div class="slider-container">
<label>静音阈值 <span id="silenceValue">0.9</span> 秒</label>
<input type="range" id="silenceSlider" min="0.1" max="5.0" step="0.1" value="0.9" oninput="updateSilence(this.value)">
</div>
<div class="section-title">🔇 噪音校准</div>
<button onclick="calibrateNoise()" class="secondary">开始校准（安静3秒）</button>
<div class="section-title">👤 声纹管理</div>
<input type="text" id="speakerName" placeholder="说话人姓名">
<button id="regBtn" onclick="registerSpeaker()">注册说话人</button>
<button onclick="deleteSpeaker()" class="danger">删除</button>
<button onclick="testSpeaker()" class="secondary">🔍 声纹测试</button>
<select id="speakerList" style="height: 380px;" onchange="document.getElementById('speakerName').value=this.value"></select>
<button onclick="refreshSpeakerList()" class="secondary">刷新列表</button>
</div>
<div class="content">
<div class="controls">
<button id="recBtn" onclick="toggleRecording()">▶️ 开始会议</button>
<button onclick="clearTranscript()" class="secondary">清屏</button>
<button onclick="saveTranscript()">💾 保存记录</button>
<span style="margin-left: auto; font-size:0.9em;" id="micStatus">🎤 麦克风未激活</span>
</div>
<div class="transcript-area" id="transcript"></div>
</div>
</div>
<script src="static/socket.io.min.js"></script>
<script>
let socket = io();
let isRecording = false;
let audioContext = null;
let mediaStream = null;
let scriptProcessor = null;
let transcriptRecords = []; // 用于保存会议记录
socket.on('connect', () => {
document.getElementById('status').innerHTML = '🟢 已连接';
refreshSpeakerList();
});
socket.on('disconnect', () => {
document.getElementById('status').innerHTML = '🔴 断开';
stopRecording();
});
socket.on('transcription', (data) => {
transcriptRecords.push(data); // 存储记录
const div = document.createElement('div');
div.className = 'msg-row';
div.innerHTML = `<span class="speaker">[${data.time}] ${data.speaker}:</span><span class="text">${escapeHtml(data.text)}</span>`;
document.getElementById('transcript').appendChild(div);
div.scrollIntoView({ behavior: 'smooth' });
});
socket.on('log', (data) => {
appendLog(data.msg);
});
socket.on('rms_update', (data) => {
const rms = data.rms;
document.getElementById('rmsText').textContent = rms.toFixed(6);
const percent = Math.min(100, (rms / 0.1) * 100);
document.getElementById('rmsFill').style.width = percent + '%';
});
socket.on('update_energy_threshold', (data) => {
document.getElementById('energySlider').value = data.value;
updateEnergy(data.value);
});
socket.on('register_result', (data) => {
alert(data.msg);
refreshSpeakerList();
document.getElementById('regBtn').disabled = false;
});
socket.on('delete_result', (data) => {
alert(data.msg);
refreshSpeakerList();
});
socket.on('speaker_list', (data) => {
const sel = document.getElementById('speakerList');
sel.innerHTML = '';
data.speakers.forEach(s => {
const opt = document.createElement('option');
opt.value = s;
opt.textContent = s;
sel.appendChild(opt);
});
});
socket.on('test_result', (data) => {
if (data.success) {
alert(`识别结果：${data.name}\\n相似度得分：${data.score}\\n详细信息：${data.msg}`);
} else {
alert('测试失败：' + data.msg);
}
});
function updateSilence(val) {
document.getElementById('silenceValue').textContent = parseFloat(val).toFixed(1);
socket.emit('set_silence_duration', { duration: val });
}
function updateEnergy(val) {
document.getElementById('energyValue').textContent = parseFloat(val).toFixed(4);
socket.emit('set_energy_threshold', { threshold: val });
}
function appendLog(msg) {
const div = document.createElement('div');
div.textContent = msg;
div.style.color = '#555';
document.getElementById('transcript').appendChild(div);
}
function escapeHtml(text) {
const d = document.createElement('div');
d.appendChild(document.createTextNode(text));
return d.innerHTML;
}
async function toggleRecording() {
if (isRecording) {
await stopRecording();
} else {
await startRecording();
}
}
async function startRecording() {
if (!socket.connected) return alert('未连接');
try {
mediaStream = await navigator.mediaDevices.getUserMedia({
audio: { channelCount: 1, echoCancellation: false, noiseSuppression: false, autoGainControl: false }
});
audioContext = new (window.AudioContext || window.webkitAudioContext)();
if (audioContext.state === 'suspended') await audioContext.resume();
const source = audioContext.createMediaStreamSource(mediaStream);
scriptProcessor = audioContext.createScriptProcessor(4096, 1, 1);
scriptProcessor.onaudioprocess = (e) => {
if (!isRecording || !socket.connected) return;
const input = e.inputBuffer.getChannelData(0);
const sampleRate = audioContext.sampleRate;
const header = new ArrayBuffer(4);
new DataView(header).setUint32(0, sampleRate, true);
const payload = new Float32Array(input);
const combined = new Uint8Array(header.byteLength + payload.byteLength);
combined.set(new Uint8Array(header), 0);
combined.set(new Uint8Array(payload.buffer), header.byteLength);
socket.emit('audio_chunk', combined.buffer);
};
source.connect(scriptProcessor);
scriptProcessor.connect(audioContext.destination);
isRecording = true;
document.getElementById('recBtn').textContent = '⏹ 停止会议';
document.getElementById('micStatus').textContent = '🎤 麦克风工作中';
socket.emit('start_recording');
updateSilence(document.getElementById('silenceSlider').value);
updateEnergy(document.getElementById('energySlider').value);
} catch (err) {
alert('无法访问麦克风: ' + err.message);
}
}
async function stopRecording() {
if (!isRecording) return;
isRecording = false;
if (scriptProcessor) {
scriptProcessor.disconnect();
scriptProcessor.onaudioprocess = null;
scriptProcessor = null;
}
if (audioContext) {
audioContext.close().catch(console.error);
audioContext = null;
}
if (mediaStream) {
mediaStream.getTracks().forEach(t => t.stop());
mediaStream = null;
}
document.getElementById('recBtn').textContent = '▶️ 开始会议';
document.getElementById('micStatus').textContent = '🎤 麦克风未激活';
socket.emit('stop_recording');
}
async function calibrateNoise() {
if (isRecording) await stopRecording();
appendLog('🎤 开始底噪校准，请保持安静 3 秒...');
let stream;
try {
stream = await navigator.mediaDevices.getUserMedia({ audio: { channelCount: 1 } });
} catch (e) {
appendLog('❌ 校准失败：无法访问麦克风');
return;
}
const actx = new (window.AudioContext || window.webkitAudioContext)();
if (actx.state === 'suspended') await actx.resume();
const src = actx.createMediaStreamSource(stream);
const node = actx.createScriptProcessor(4096, 1, 1);
const samples = [];
let duration = 0;
node.onaudioprocess = (e) => {
const data = e.inputBuffer.getChannelData(0);
samples.push(new Float32Array(data));
duration += data.length / actx.sampleRate;
if (duration >= 3.0) {
node.disconnect();
src.disconnect();
actx.close();
stream.getTracks().forEach(t => t.stop());
const totalLen = samples.reduce((s, a) => s + a.length, 0);
const merged = new Float32Array(totalLen);
let off = 0;
samples.forEach(a => { merged.set(a, off); off += a.length; });
let sumSq = 0;
for (let i = 0; i < merged.length; i++) sumSq += merged[i] * merged[i];
const rms = Math.sqrt(sumSq / merged.length);
socket.emit('set_noise_threshold', { rms: rms });
appendLog(`✅ 校准完成，底噪 RMS: ${rms.toFixed(6)}`);
}
};
src.connect(node);
node.connect(actx.destination);
}
function clearTranscript() {
document.getElementById('transcript').innerHTML = '';
transcriptRecords = []; // 同时清空记录数组
}
function registerSpeaker() {
const name = document.getElementById('speakerName').value.trim();
if (!name) return alert('请输入姓名');
if (isRecording) return alert('请先停止会议');
const regBtn = document.getElementById('regBtn');
regBtn.disabled = true;
regBtn.textContent = '录制中（3秒）...';
navigator.mediaDevices.getUserMedia({ audio: { channelCount: 1 } })
.then(async stream => {
const actx = new AudioContext();
if (actx.state === 'suspended') {
await actx.resume();
}
const src = actx.createMediaStreamSource(stream);
const recorder = actx.createScriptProcessor(4096, 1, 1);
const chunks = [];
let dur = 0;
recorder.onaudioprocess = (e) => {
chunks.push(new Float32Array(e.inputBuffer.getChannelData(0)));
dur += e.inputBuffer.duration;
if (dur >= 3) {
recorder.disconnect();
src.disconnect();
actx.close();
stream.getTracks().forEach(t => t.stop());
const total = chunks.reduce((s,a) => s + a.length, 0);
const merged = new Float32Array(total);
let off = 0;
chunks.forEach(a => { merged.set(a, off); off += a.length; });
const int16 = new Int16Array(merged.length);
for (let i = 0; i < merged.length; i++) {
const s = Math.max(-1, Math.min(1, merged[i]));
int16[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
}
const wavBuffer = encodeWAV(int16, actx.sampleRate);
const blob = new Blob([wavBuffer], { type: 'audio/wav' });
const reader = new FileReader();
reader.onloadend = function() {
const base64data = reader.result.split(',')[1];
socket.emit('register_speaker', { name, audio: base64data });
regBtn.textContent = '注册说话人';
};
reader.readAsDataURL(blob);
}
};
src.connect(recorder);
recorder.connect(actx.destination);
})
.catch(err => {
alert('麦克风错误: ' + err.message);
regBtn.disabled = false;
regBtn.textContent = '注册说话人';
});
}
function deleteSpeaker() {
const name = document.getElementById('speakerName').value.trim();
if (!name) return alert('请输入姓名');
socket.emit('delete_speaker', { name });
}
function refreshSpeakerList() {
socket.emit('get_speaker_list');
}
function testSpeaker() {
if (isRecording) {
alert('请先停止会议录音');
return;
}
appendLog('🔍 开始声纹测试，请说话 3 秒...');
navigator.mediaDevices.getUserMedia({ audio: { channelCount: 1 } })
.then(async stream => {
const actx = new AudioContext();
if (actx.state === 'suspended') {
await actx.resume();
}
const src = actx.createMediaStreamSource(stream);
const recorder = actx.createScriptProcessor(4096, 1, 1);
const chunks = [];
let dur = 0;
recorder.onaudioprocess = (e) => {
chunks.push(new Float32Array(e.inputBuffer.getChannelData(0)));
dur += e.inputBuffer.duration;
if (dur >= 3) {
recorder.disconnect();
src.disconnect();
actx.close();
stream.getTracks().forEach(t => t.stop());
const total = chunks.reduce((s,a) => s + a.length, 0);
const merged = new Float32Array(total);
let off = 0;
chunks.forEach(a => { merged.set(a, off); off += a.length; });
const int16 = new Int16Array(merged.length);
for (let i = 0; i < merged.length; i++) {
const s = Math.max(-1, Math.min(1, merged[i]));
int16[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
}
const wavBuffer = encodeWAV(int16, actx.sampleRate);
const blob = new Blob([wavBuffer], { type: 'audio/wav' });
const reader = new FileReader();
reader.onloadend = function() {
const base64data = reader.result.split(',')[1];
socket.emit('test_speaker', { audio: base64data });
};
reader.readAsDataURL(blob);
}
};
src.connect(recorder);
recorder.connect(actx.destination);
})
.catch(err => {
alert('麦克风错误: ' + err.message);
});
}
// 保存会议记录到本地文件
function saveTranscript() {
if (transcriptRecords.length === 0) {
alert('暂无会议记录可保存');
return;
}
let content = '';
transcriptRecords.forEach(record => {
content += `[${record.time}] ${record.speaker}: ${record.text}\\n`;
});
const blob = new Blob([content], { type: 'text/plain;charset=utf-8' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
const now = new Date();
const filename = `会议记录_${now.getFullYear()}-${(now.getMonth()+1).toString().padStart(2,'0')}-${now.getDate().toString().padStart(2,'0')}_${now.getHours().toString().padStart(2,'0')}-${now.getMinutes().toString().padStart(2,'0')}.txt`;
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
appendLog('✅ 会议记录已保存为 ' + filename);
}
function encodeWAV(samples, sampleRate) {
const buffer = new ArrayBuffer(44 + samples.length * 2);
const view = new DataView(buffer);
writeString(view, 0, 'RIFF');
view.setUint32(4, 36 + samples.length * 2, true);
writeString(view, 8, 'WAVE');
writeString(view, 12, 'fmt ');
view.setUint32(16, 16, true);
view.setUint16(20, 1, true);
view.setUint16(22, 1, true);
view.setUint32(24, sampleRate, true);
view.setUint32(28, sampleRate * 2, true);
view.setUint16(32, 2, true);
view.setUint16(34, 16, true);
writeString(view, 36, 'data');
view.setUint32(40, samples.length * 2, true);
for (let i = 0; i < samples.length; i++) {
view.setInt16(44 + i * 2, samples[i], true);
}
return buffer;
}
function writeString(view, offset, string) {
for (let i = 0; i < string.length; i++) {
view.setUint8(offset + i, string.charCodeAt(i));
}
}
// 初始化滑块数值显示
updateSilence(document.getElementById('silenceSlider').value);
updateEnergy(document.getElementById('energySlider').value);
refreshSpeakerList();
</script>
</body>
</html>
'''
@app.route('/')
def index():
return render_template_string(HTML_TEMPLATE)
if __name__ == '__main__':
print("🚀 智能会议记录系统（Qwen3-ASR，声纹优化版，含声纹测试与记录保存）启动", flush=True)
print(" 访问地址: http://0.0.0.0:7898", flush=True)
socketio.run(app, host='0.0.0.0', port=7898, debug=False)3.3模型下载与准备
1、Qwen3-ASR-1.7B（语音识别模型）：
下载链接：https://modelscope.cn/models/Qwen/Qwen3-ASR-1.7B/files
2、iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common（说话人确认模型）：
下载链接：https://modelscope.cn/models/iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common/files
3.4模型启动参数
1、Qwen3-ASR-1.7B启动参数：
cat Qwen3-ASR-1.7B.sh
HIP_VISIBLE_DEVICES=6 vllm serve '/home/models/Qwen3-ASR-1.7B' \
--trust-remote-code \
--gpu-memory-utilization 0.3 \
--limit-mm-per-prompt '{"audio": 1}' \
--port 8084 \
--served-model-name Qwen3-ASR-1.7B \
--api-key PassWord@1234562、iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common启动参数：
SPEAKER_MODEL_ID = "iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common"
SPEAKER_DB_PATH = "speaker_db.json"
# ================== 声纹管理 ==================
class SpeakerManager:
def __init__(self, db_path=SPEAKER_DB_PATH):
self.db_path = db_path
self.db = self.load_db()
self.lock = threading.Lock()
self.device = "cuda:6" if torch.cuda.is_available() else "cpu"
self.speaker_model = Model.from_pretrained(
SPEAKER_MODEL_ID,
revision=None,
device=self.device,
)
self.speaker_model.eval()
print("✅ 声纹模型加载成功", flush=True)四、运行测试
1、启动Qwen3-ASR-1.7B
docker exec -it qwen3-asr bash
cd /home/models/
nohup ./Qwen3-ASR-1.7B.sh &2、启动智能会议记录系统自编程序
python 会议记录系统.py
/home/models/会议记录系统.py:8: EventletDeprecationWarning:
Eventlet is deprecated. It is currently being maintained in bugfix mode, and
we strongly recommend against using it for new projects.
If you are already using Eventlet, we recommend migrating to a different
framework. For more detail see
https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
import eventlet
Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common
2026-07-11 16:09:15,218 - modelscope - INFO - initialize model from /root/.cache/modelscope/hub/models/iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common
✅ 声纹模型加载成功
✅ ASR API 就绪
🚀 智能会议记录系统（Qwen3-ASR，声纹优化版，含声纹测试与记录保存）启动
访问地址: http://0.0.0.0:78983、访问智能会议记录系统自编程序
用谷歌浏览器访问，访问之前需要做安全授权设置：
地址栏执行：chrome://flags/
搜索：Insecureoriginstreatedassecure
填入：http://192.168.222.65:7898
然后访问：http://192.168.222.65:7898
先执行噪音校准，获取背景噪声。根据说话人语速调整VAD静音持续时间，语速越快，该时间就越小。注册说话人、声纹测试，最后开始会议。
欢迎加入DeepSeek 技术社区。在这里，你可以找到志同道合的朋友，共同探索AI技术的奥秘。
更多推荐
72
0- 0
已为社区贡献3条内容
所有评论(0)