【中文字幕】2017春季CS231n 斯坦福深度视觉识别课

import numpy as np np.random.seed(0) N,D=3,4 x=np.random.randn(N,D) y=np.random.randn(N,D) z=np.random.randn(N,D) a=x*y b=a+z c=np.sum(b) grad_c=1.0 grad_b=grad_c*np.ones((N,D)) grad_a=grad_b.copy() grad_z=grad_b.copy() grad_x=grad_a*y grad_y=grad_a*x

challenges:

• viewpoint variantion
• illumination（照明）
• deformation（可变形性）
• occlusion(遮挡)
• background clutter（物体和背景难以区分）

 

CS331  and  CS431 for advanced topics in computer vision

# Vanilla Gradient Descent while True: weights_grad=evaluate_gradient(loss_fun,data,weights) weights+=-step_size*weights_grad # perform parameter update # Adam (almost) first_moment=0 second_moment=0 while True: dx=compute_gradient(x) first_moment=beta1*first_moment+(1-beta1)*dx second_moment=beta2*second_moment+(1-beta2)*dx*dx x-=learning_rate*first_moment/(np.sqrt(second_moment)+1e-7) while True: data_batch=dataset.sample_data_batch() loss=network.forwadr(data_batch) dx=network.backward() x+=-learning_rate*dx x_test=0.995*x_test+0.005*x # use for test set

# Double check taht the loss is reasonable: def init_two_layer_model(input_size,hidden_size,output_size): # initialize a model model={} model['W1']=0.0001*np.random.randn(input_size,hidden_size) model['b1']=np.zeros(hidden_size) model['W2']=0.0001*np.random.randn(hidden_size,output_size) model['b2']=np.zeros(output_size) return model model=init_two_layer_model(32*32*3,50,10) #input size,hidden size,number of classes loss,grad=two_layer_net(X_train,model,y_train,0.0) # 0.0 denotes disable regularization print loss """ 如果我们加上了正则项，loss会变大一点，这是完整性 """ """ 往往需要先确保可以过拟合非常小部分的训练数据""" model=init_two_layer_model(32*32*3,50,10) # trainer=ClassfierTrain() X_tiny=x_train[:20] # take examples y_tiny=y_train[:20] best_model,stats=trainer.train(X_tiny,y_tiny,X_tiny,y_tiny, model,two_layer_net, num_epochs=200,reg=0.0, updat='sgd',learning_rate_decay=1, sample_batchs=False, learning_rate=1e-3,verbose=True)

 

1. loss not going down:

learning rate too low

2. 即使损失没怎么变化，但正确率却很快上升到了20%，为什么？

虽然这里的分布依然很分散，因为我们的损失项很接近，但是当我们移动这些分布时，都朝着正确的方向在轻微的移动，我们的权重参数在朝着正确的方向改变，现在准确率可能发生突变，因为我们正选取最大的准确率，所以准确率会得到一个很大的提升，虽然损失还是很分散。

3.loss exploding:

learning rate too high

 

 

# Double check taht the loss is reasonable

http://cs.standard.edu/people/karpathy/convnetjs/demo/cifar10.html

import numpy as np from numpy.random import randn N,D_in,H,D_out=64,1000,100,10 x,y=randn(N,D_in),randn(N,D_out) w1,w2=randn(D_in,H),randn(H,D_out) for t in range(2000): h=1/(1+np.exp(-x.dot(w1))) y_pred=h.dot(w2) loss=np.square(y_pred-y).sum() print(t,loss) grad_y_pred=2.0*(y_pred-y) grad_w2=h.T.dot(grad_y_pred) grad_h=grad_y_pred.dot(w2.T) grad_w1=x.T.dot(grad_h*h*(1-h)) w1-=1e-4*grad_w1 w2-=1e-4*grad_w2 class Neuron: # ... def neuron_tick(inputs): """ assume inputs and weights are 1-D numpy arrays and bias is a number""" cell_body_sum=np.sum(inputs*self.weights)+self.bias firing_rate=1.0/(1.0+math.exp(-cell_body_sum)) return firing_rate # forward-pass of a 3-layer neural network: f=lambda x:1.0/(1.0+np.exp(-x)) x=np.random.randn(3,1) h1=f(np.dot(W1,x)+b1) h2=f(np.dot(W2,h1)+b2) out=np.dot(W3,h2)+b3

交叉验证 超参数

线性分类

损失函数